-- Logs begin at Wed 2022-05-25 13:53:00 UTC, end at Wed 2022-05-25 15:34:35 UTC. -- May 25 14:25:00.820557 dv-cinder-815746-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 14:25:00.829729 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 14:25:00 2022] *** May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: nodename: dv-cinder-815746-6-iscsi May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: machine: x86_64 May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: clock source: unix May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: pcre jit disabled May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: detected number of CPU cores: 4 May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: current working directory: / May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: detected binary path: /usr/bin/uwsgi-core May 25 14:25:00.831782 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: your processes number limit is 47680 May 25 14:25:00.832361 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: your memory page size is 4096 bytes May 25 14:25:00.832361 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: detected max file descriptor number: 2048 May 25 14:25:00.832361 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: lock engine: pthread robust mutexes May 25 14:25:00.832361 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: thunder lock: enabled May 25 14:25:00.832361 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 14:25:00.832361 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 14:25:00.876259 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: Python main interpreter initialized at 0x56255b2829c0 May 25 14:25:00.876259 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: python threads support enabled May 25 14:25:00.876259 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: your server socket listen backlog is limited to 100 connections May 25 14:25:00.876259 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: your mercy for graceful operations on workers is 90 seconds May 25 14:25:00.876813 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: mapped 403077 bytes (393 KB) for 2 cores May 25 14:25:00.876813 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: *** Operational MODE: preforking *** May 25 14:25:00.877023 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: *** uWSGI is running in multiple interpreter mode *** May 25 14:25:00.877023 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: spawned uWSGI master process (pid: 80349) May 25 14:25:00.877191 dv-cinder-815746-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 14:25:00.883218 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: spawned uWSGI worker 1 (pid: 80395, cores: 1) May 25 14:25:00.883218 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: spawned uWSGI worker 2 (pid: 80396, cores: 1) May 25 14:25:00.883218 dv-cinder-815746-6-iscsi devstack@keystone.service[80349]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 14:25:06.489590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:25:06.489590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 25 14:25:06.490461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:25:06.490461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 25 14:25:06.509611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:25:06.509611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: from cryptography.utils import int_from_bytes May 25 14:25:06.510252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 14:25:06.510252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: from cryptography.utils import int_from_bytes May 25 14:25:06.539606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 14:25:06.539803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 14:25:06.539969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] command line args: [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 14:25:06.540240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 14:25:06.540372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ================================================================================ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 14:25:06.540651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.540892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_dir = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.541067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.541256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_source = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.541387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] debug = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.541612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_config_append = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_dir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.543845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.544018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.544268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.544572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.544697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.544840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.545158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.545340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.545623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.545944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.546106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.546241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.546918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.546918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.546918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.548557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.32/identity {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.548557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] publish_errors = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.548557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.548557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.548557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.548557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] standard_threads = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_journal = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_json = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.550053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_stderr = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.551220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_syslog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.551220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.551220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.551220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.551220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.551673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.551673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.551884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.552400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.552400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.552594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.553295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.553452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.553710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.553951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.554210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.554565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.554724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.555008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.555320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.555586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.555902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.555902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.556465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.556616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.556828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.557075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.557320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.557486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.557830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.558115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 14:25:06.558287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.559008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 14:25:06.559008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] command line args: [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 14:25:06.560042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.560315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 14:25:06.561432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.562249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ================================================================================ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 14:25:06.562453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.562708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] admin_token = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.562889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.563050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_dir = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.563206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.563369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.563487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.563487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.563749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_source = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.563912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.564053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] debug = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.564208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.564208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.564429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.564630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.564710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.564710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.564953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.565119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.565213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.565213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.565423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.565580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.565720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.565813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.565972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_config_append = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.565972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.566260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.566517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_dir = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.566678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.566950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.567103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.567210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.567326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.567473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.567691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.567834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.567948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.568064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.568198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.568328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.568328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.568588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.568727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.569086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.569086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.569309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.569454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.569580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.569580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.569798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.569923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.570033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.570195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.570329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.570458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.570666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.32/identity {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.570918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.570990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] publish_errors = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.571129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.571399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.571536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.571669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.571810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] standard_threads = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.571941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.571941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.572132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.572267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_journal = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_json = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.572459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_stderr = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.572662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_syslog = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.572804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 14:25:06.572872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.572939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.573992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.574065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.574132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.574195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.574262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.574320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.574381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.575992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.576910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.577908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.external = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.password = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.578981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.token = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579754 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.579883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.580864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.581875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.582951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.583947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584153 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.584937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.585926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.586992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.587928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.588976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.589949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.590921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.591902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.592931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.external = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.593973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.password = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.token = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.594966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.595931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.596944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.597985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.598953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.599866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.600952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.601952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.602915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.603982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.604976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.605100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.605312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.605430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.605547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.605795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.605933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.606964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.607944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.608914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.609902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.610897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.611890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.612988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.613941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.614900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.615886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.616873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.617996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.618990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.619109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.619507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.619632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.619762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.619884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620005 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.620822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.621972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.622802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.623981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.624982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.625985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.626068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.626166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.626245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.626315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 14:25:06.626406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.626638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.626902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.627144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.627438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.627681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.628102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.628333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.628576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.628817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.629058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.629299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.629524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.629768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.630012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.630251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.630478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.630707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.630957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.631200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.631448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.631687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.631917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.632144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.632374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.632600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.632830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.633069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.633298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.633547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.633785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.634015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.634252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.634488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.634738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.634991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.635225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.635454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.635694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.635935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.636160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.636393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.636688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.636952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.637192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.637499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.637732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.637966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.638194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.638423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.638652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.638908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.639148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.639375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.639612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.639838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.640078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.640305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.640540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.640762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.640994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.641220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 14:25:06.641384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 14:25:06.657929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.658131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.658366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.666552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.666745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.667016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.667141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.677374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.696378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.696595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.703332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.703543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.751188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.753844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.803752 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.803982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.804818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.805057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.871910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.872069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.872179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.872281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.872529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.872679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.895781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.896987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.897103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.897184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.908854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.909020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.921594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.921869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.921986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.922924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.923003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.923135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.923216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.923216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 14:25:06.941301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.941540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:25:06.941654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.941739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:25:06.941800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.942007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:25:06.942133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.942255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.942353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 14:25:06.942476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.942556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.942737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.942898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.943056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.943140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.943286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.954374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.954648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.954932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.955208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.955851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.956130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.956392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.956667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.963362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:06.964752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:06.966683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:06.966983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.967272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.967509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.967809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:06.968339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.968998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.969120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.969272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.969478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.969615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.969775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.970011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.970277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.970544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.970852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.995950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.997215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.997718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:06.997990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:06.999245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:06.999520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.003391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.003608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.003881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.004828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.005100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.005366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.011220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.011496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.012764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.013040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.016067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.016468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.016745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.017020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.017294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.017553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.017681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.017822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.017916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.018040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.018193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.018491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.018781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.019077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.019351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.019624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.036799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.037070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.038420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.038689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.041855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.043404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.045570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.045860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.047106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.047395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.050898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.052421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.054844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.056360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.058597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.058946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.059257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.059541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.059797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.060117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.060244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.060477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.060750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.061024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.061305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.061577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.079052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.079338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.079599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.079861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.080564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.080844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.081102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.081356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.087874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.089450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.089583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.089854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.091225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.091499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.092958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.093237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.093518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.093792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.094065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.094526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.094788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.095094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.095412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.095688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.107592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.107846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.108150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.108431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.108704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.108971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.109160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.109275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.109444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.109739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.110019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.110292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.110555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.110866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.127358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.128881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.130701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.132150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.134024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.135498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.135699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.137278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.137430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.137589 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.137869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.139096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.139428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.139717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.145367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.147000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.147195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.148724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.150234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.150546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.150852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.151128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.151981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.152261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.152534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.152776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.160255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.160543 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.160936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.161878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.162157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.162534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.168967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.169250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.169520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.169792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.170058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.170642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.170860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 14:25:07.170959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.171978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.172885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.173115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.173360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.173588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.173873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 14:25:07.213263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56255b2829c0 pid: 80396 (default app) May 25 14:25:07.215824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56255b2829c0 pid: 80395 (default app) May 25 14:25:07.216709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094adc0a-3091-4f64-8101-a82131dfaa5c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:07.216993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094adc0a-3091-4f64-8101-a82131dfaa5c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:07.217263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094adc0a-3091-4f64-8101-a82131dfaa5c None None] PATH_INFO: `/v3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:07.221581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1/1] 10.222.0.32 () {50 vars in 791 bytes} [Wed May 25 14:25:07 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:15.223442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:15.224349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:15.224349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:15.231951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1/2] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:25:15.238380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:15.238979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:15.238979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:15.306708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: INFO dbcounter [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] Registered counter for database keystone May 25 14:25:15.329234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG oslo_db.sqlalchemy.engines [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 14:25:15.336800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writer thread running {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 14:25:15.530733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG passlib.handlers.bcrypt [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80396) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 14:25:15.531060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG passlib.handlers.bcrypt [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80396) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 14:25:15.589238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:15.590940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:15.746974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 2/3] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 14:25:15.895987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:15.968981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: INFO dbcounter [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] Registered counter for database keystone May 25 14:25:15.990622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG oslo_db.sqlalchemy.engines [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 14:25:15.998268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writer thread running {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 14:25:16.117996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29a8d0-4c4c-44a7-b992-eb22875ac87b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:16.122897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:16.123570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:16.123856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:16.124126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:16.133423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:16.184562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING keystone.common.rbac_enforcer.enforcer [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 14:25:16.184910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558936e8-1644-4343-bd8d-ea9574f70258 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:16.196062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 2/4] 10.222.0.32 () {58 vars in 1324 bytes} [Wed May 25 14:25:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:25:16.202602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:16.217741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbda2f8-bca2-4c9d-bd7b-466921785113 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:16.222784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:16.223947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:16.224252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:16.224519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] PATH_INFO: `/v3/projects/9157b1edcbb3430198b55709b5ddfe19` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:16.226093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] RBAC: Authorizing `identity:get_project(project_id=9157b1edcbb3430198b55709b5ddfe19)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:16.278135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING keystone.common.rbac_enforcer.enforcer [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 14:25:16.278868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de584972-b2bd-4cb1-9f01-c3bc3d10b3e8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:16.283251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 3/5] 10.222.0.32 () {56 vars in 1207 bytes} [Wed May 25 14:25:16 2022] GET /identity/v3/projects/9157b1edcbb3430198b55709b5ddfe19 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:18.483664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fb6db-668a-47ae-b33d-4e8d7d356f68 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:18.484229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fb6db-668a-47ae-b33d-4e8d7d356f68 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:18.484229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fb6db-668a-47ae-b33d-4e8d7d356f68 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:18.488375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 3/6] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:18.497531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:18.498007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:18.498109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:18.521637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:18.523255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:18.533713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 4/7] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 14:25:18.558219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fb6db-668a-47ae-b33d-4e8d7d356f68 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:18.572267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fb6db-668a-47ae-b33d-4e8d7d356f68 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:18.579547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13c0ed-66c7-44a9-b5a9-6291bf862fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:18.580612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c0ed-66c7-44a9-b5a9-6291bf862fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:18.581062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c0ed-66c7-44a9-b5a9-6291bf862fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:18.581490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13c0ed-66c7-44a9-b5a9-6291bf862fb8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:18.594994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13c0ed-66c7-44a9-b5a9-6291bf862fb8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:18.596803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13c0ed-66c7-44a9-b5a9-6291bf862fb8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:18.607935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 4/8] 10.222.0.32 () {58 vars in 1324 bytes} [Wed May 25 14:25:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:25:18.615258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:18.628496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646f939-06d0-4883-983f-f10199fe5920 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:18.635653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea18a5f-0038-439e-88ca-66cb53dbc098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:18.636592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18a5f-0038-439e-88ca-66cb53dbc098 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:18.636924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18a5f-0038-439e-88ca-66cb53dbc098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:18.637202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea18a5f-0038-439e-88ca-66cb53dbc098 admin admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:18.638962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18a5f-0038-439e-88ca-66cb53dbc098 admin admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:18.641120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea18a5f-0038-439e-88ca-66cb53dbc098 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:18.642768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 5/9] 10.222.0.32 () {56 vars in 1198 bytes} [Wed May 25 14:25:18 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:20.955355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bcec2-3d8e-494a-ab7d-0144a5b7d999 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:20.955903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bcec2-3d8e-494a-ab7d-0144a5b7d999 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:20.955903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bcec2-3d8e-494a-ab7d-0144a5b7d999 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:20.957280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 5/10] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:20.963404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:20.963916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:20.963916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:20.981394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb11d5-71ee-4fc0-8a76-413139f69d4c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:20.981672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb11d5-71ee-4fc0-8a76-413139f69d4c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:20.981946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb11d5-71ee-4fc0-8a76-413139f69d4c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:20.982105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:20.983435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 6/11] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:25:20.983582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:20.988990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:20.989267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:20.989533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.031635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80395) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 14:25:21.031898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80395) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 14:25:21.072199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 6/12] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:21.079020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:21.080649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:21.099000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.124280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:21.187616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 7/13] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:21.209823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.211694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836a4bc-4cf5-462d-8375-1f35aa488614 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:21.249464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:21.250735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:21.251228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:21.251656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.266044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:21.277758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:21.278971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:21.279781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 7/14] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:25:21.288051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.438715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4490233-0e9a-44ea-9a8c-29a507226ec9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:21.442032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:21.480465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:21.481397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:21.481679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:21.481957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.496874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:21.519035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099682a7-320a-4d84-9e0d-fb0149914df2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:21.554272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:21.554974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:21.555254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:21.555525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.556288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:21.556587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:21.558162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:21.572590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 8/15] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:21.572945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:21.575292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:21.579299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.689371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956173a-7a49-40d6-b6da-644f622b1440 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:21.703536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 8/16] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:21.709391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.712488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:21.730409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:21.731827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:21.732277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:21.732698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.738700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:21.740224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:21.741891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 9/17] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:21.747510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.763305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:21.825922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7b86c-e9d5-4d9d-887c-e5de9775d9a6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:21.864622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:21.864963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf128743-276f-45b0-90b1-fac8fd85a480 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:21.865444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:21.865721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:21.865989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.890641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:21.892155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:21.904685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:21.905504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:21.905785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:21.906052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:21.907342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:21.908829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:21.910450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 10/18] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:25:21.916203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:21.917067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:21.918089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:25:21.918089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:25:21.918089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:25:21.918089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:21.918313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.918378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 9/19] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:21.925823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:21.985353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:22.016763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:22.030767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba34ada-0037-4762-bf08-3c942612f697 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:22.042422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daae669-6d43-498d-b34e-507de6b1c8a0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:22.066424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:22.068293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:22.068293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:22.068293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:22.068539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:22.068687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:22.070105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f8a41-c02e-41f7-99ca-ca38b752407f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:22.081884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:22.082564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:22.082878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:22.083154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:22.084005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.domain.id=default, user.id=73561edcff5e424d9ae49ab57406b79a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:22.084316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:22.086454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e99587-0fec-49f8-8459-bbce0470e487 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:22.088233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 10/20] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:22.094500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 11/21] 10.222.0.32 () {56 vars in 1322 bytes} [Wed May 25 14:25:21 2022] GET /identity/v3/role_assignments?user.id=73561edcff5e424d9ae49ab57406b79a&scope.domain.id=default&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 233 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:24.882876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c33e2-b9fe-46d3-8056-b840ab972152 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:24.882876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c33e2-b9fe-46d3-8056-b840ab972152 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:24.882876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c33e2-b9fe-46d3-8056-b840ab972152 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:24.882876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 11/22] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:24.889737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7cb50-85ed-4030-a84e-3c2406584fe2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:24.890273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7cb50-85ed-4030-a84e-3c2406584fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:24.890273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7cb50-85ed-4030-a84e-3c2406584fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:24.917205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a9f7cb50-85ed-4030-a84e-3c2406584fe2 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:24.918436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a9f7cb50-85ed-4030-a84e-3c2406584fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:25.021825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 12/23] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:25.056474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c33e2-b9fe-46d3-8056-b840ab972152 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:25.079803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e337ea5-142d-497c-9324-dc31615b2bc2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.080426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e337ea5-142d-497c-9324-dc31615b2bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.080898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e337ea5-142d-497c-9324-dc31615b2bc2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.082774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 13/24] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:25.097551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e260-e8ef-40f7-a15e-a4b3e96914fb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.098216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e260-e8ef-40f7-a15e-a4b3e96914fb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.098706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6e260-e8ef-40f7-a15e-a4b3e96914fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.135894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aed6e260-e8ef-40f7-a15e-a4b3e96914fb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:25.137466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aed6e260-e8ef-40f7-a15e-a4b3e96914fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:25.219261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c33e2-b9fe-46d3-8056-b840ab972152 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:25.240668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:25.283296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:25.284119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.284399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.284674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.302210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 14/25] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:25.304003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f1cad-b0db-454c-b483-d4bdda1d4edb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.304292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f1cad-b0db-454c-b483-d4bdda1d4edb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.304567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f1cad-b0db-454c-b483-d4bdda1d4edb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.305885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 15/26] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:25.312265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.312515 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.312734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.314989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:25.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:25.325104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:25.325849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:25.328180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 12/27] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:25.331113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:25.338267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:25.355992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:25.357249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:25.469797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 16/28] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:25.477024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:25.501339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e11b7-5b90-4236-ac2b-9079fd5e2ca2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:25.540857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:25.541786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.542203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.542680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.543705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:25.544174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:25.545802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233007-2fbc-48a1-8c9f-e51927dd6402 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:25.559966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 13/29] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:25.561839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591cca9-7362-42ec-9f83-eb0abff5a972 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.562337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591cca9-7362-42ec-9f83-eb0abff5a972 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.562865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591cca9-7362-42ec-9f83-eb0abff5a972 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.564250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 14/30] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:25.565215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591cca9-7362-42ec-9f83-eb0abff5a972 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:25.609347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcbcef-5a5f-482b-baff-2abbae423256 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:25.615307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:25.649840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:25.651062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.651484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.651895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.654306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:25.665567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:25.667686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:25:25.688691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:25.689709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:25.690695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:25:25.690695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:25:25.690695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:25:25.690695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:25.691894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591cca9-7362-42ec-9f83-eb0abff5a972 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:25.692950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 17/31] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:25.693936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:25.747689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:25.749092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.749379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.749652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.750493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:25.752549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:25.851101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e41d9-2172-45a8-864d-f52fd9d91dbd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:25.889340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b4d41-ef24-4a10-b35c-edea17e1729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:25.890296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b4d41-ef24-4a10-b35c-edea17e1729b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.890862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b4d41-ef24-4a10-b35c-edea17e1729b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.890862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b4d41-ef24-4a10-b35c-edea17e1729b None admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.892110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b4d41-ef24-4a10-b35c-edea17e1729b None admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:25.893638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b4d41-ef24-4a10-b35c-edea17e1729b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:25.895373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 18/32] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:25 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:25.897013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:25.897463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:25.897884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:25.905528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 15/33] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 25 14:25:25 2022] POST /identity/v3/roles => generated 219 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:25.915550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:25.919661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:25.926808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:25.991108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:26.015158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 19/34] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:26.016635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:26.121076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:26.127367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524caf1-d922-4f50-96ef-89ce1d85d59d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:26.150383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cf59e-9cf6-401b-870b-0acde37cbee4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:26.170166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:26.170948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.171232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.171509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.172234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:26.172537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:26.174075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:26.188850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 16/35] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:26.189480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:26.201387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf130cf-59e9-4c3d-95c3-a5cfefa7fed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:26.202800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf130cf-59e9-4c3d-95c3-a5cfefa7fed6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.203678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf130cf-59e9-4c3d-95c3-a5cfefa7fed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.203678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf130cf-59e9-4c3d-95c3-a5cfefa7fed6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.205870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf130cf-59e9-4c3d-95c3-a5cfefa7fed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:26.209239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf130cf-59e9-4c3d-95c3-a5cfefa7fed6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:26.211172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 20/36] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:26.213280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d6396-f316-4634-8817-0965b2475141 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.213746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d6396-f316-4634-8817-0965b2475141 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.214202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d6396-f316-4634-8817-0965b2475141 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.217678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 21/37] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:26.218523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d6396-f316-4634-8817-0965b2475141 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:26.285311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:26.322946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6ab4e-76f3-4462-b801-e5ad6ba58c28 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:26.343544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d6396-f316-4634-8817-0965b2475141 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:26.360710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:26.361489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.361718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.361988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.363042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:26.364513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:26.381990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b52e8-382b-4f7a-b24c-7ebf1233cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:26.382635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b52e8-382b-4f7a-b24c-7ebf1233cc76 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.382948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b52e8-382b-4f7a-b24c-7ebf1233cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.383222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b52e8-382b-4f7a-b24c-7ebf1233cc76 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.395263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b52e8-382b-4f7a-b24c-7ebf1233cc76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:26.395459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 17/38] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 25 14:25:26 2022] POST /identity/v3/roles => generated 225 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:26.395459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:26.395625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b52e8-382b-4f7a-b24c-7ebf1233cc76 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:26.426159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:26.543046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19217c-7d9e-4a9b-820d-428429e08c57 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:26.578677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 22/39] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:25:26 2022] POST /identity/v3/projects => generated 301 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:26.580473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.580766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.581042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.590714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cc93f-c63e-4656-8760-6897416baac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:26.591672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cc93f-c63e-4656-8760-6897416baac5 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.592021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cc93f-c63e-4656-8760-6897416baac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.592373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cc93f-c63e-4656-8760-6897416baac5 None admin] PATH_INFO: `/v3/domains/default/users/73561edcff5e424d9ae49ab57406b79a/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.595296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cc93f-c63e-4656-8760-6897416baac5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=73561edcff5e424d9ae49ab57406b79a, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:26.596792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cc93f-c63e-4656-8760-6897416baac5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:26.605632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:26.608042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:26.622650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 18/40] 10.222.0.32 () {58 vars in 1361 bytes} [Wed May 25 14:25:26 2022] PUT /identity/v3/domains/default/users/73561edcff5e424d9ae49ab57406b79a/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:26.679487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741f856-14a2-476c-9da1-919124af397e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.679670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741f856-14a2-476c-9da1-919124af397e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.680071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741f856-14a2-476c-9da1-919124af397e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.681576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 19/41] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:25:26.687855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.688142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.688423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.705158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 23/42] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:26.721479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:26.723596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:26.730818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:26.781052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:26.844888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 20/43] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:26.870575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:26.879687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128f0a4-000f-40f3-8109-59ee90be8021 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:26.886650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:26.927318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:26.928366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:26.928659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:26.928937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:26.929771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:26.931363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:26.958900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 24/44] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 25 14:25:26 2022] POST /identity/v3/roles => generated 223 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:25:27.015465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07d2a-6e49-4b20-87f5-bfd43a279097 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:27.069377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cdf053-cf10-43bb-85d9-d969bf4db059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:27.070188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdf053-cf10-43bb-85d9-d969bf4db059 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:27.070438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdf053-cf10-43bb-85d9-d969bf4db059 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:27.070739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cdf053-cf10-43bb-85d9-d969bf4db059 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:27.072083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cdf053-cf10-43bb-85d9-d969bf4db059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:27.073592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cdf053-cf10-43bb-85d9-d969bf4db059 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:27.075560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 21/45] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:27.085159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:27.200870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:27.209486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b6247-e393-4ecc-92e3-8da87b7a8941 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:27.252723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e8cba-6cb8-4211-a465-d103306d80da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:27.253399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e8cba-6cb8-4211-a465-d103306d80da None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:27.253678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e8cba-6cb8-4211-a465-d103306d80da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:27.253944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e8cba-6cb8-4211-a465-d103306d80da None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:27.254653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e8cba-6cb8-4211-a465-d103306d80da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:27.256176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e8cba-6cb8-4211-a465-d103306d80da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:27.282506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 25/46] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:25:27 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:28.918404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99139562-f9a2-49b8-97a1-11f3802b7712 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:28.919014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99139562-f9a2-49b8-97a1-11f3802b7712 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:28.919014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99139562-f9a2-49b8-97a1-11f3802b7712 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:28.920399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 22/47] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:28.926060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:28.926525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:28.926704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:28.945767 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:28.947026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:29.028652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 26/48] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:29.052009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99139562-f9a2-49b8-97a1-11f3802b7712 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.139873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:29.168439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99139562-f9a2-49b8-97a1-11f3802b7712 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.203863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.204671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.204953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.205219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.216679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:29.224630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:29.225512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:29.226464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 23/49] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:29.232050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.314734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:29.345856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80661d57-473c-4eb4-9c9a-0148598cd183 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.383378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.384062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.384349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.384624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.385394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:29.385700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:29.387149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97b4b5-0291-470f-94ff-cbf0201a0fad None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:29.396609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 27/50] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:29.402412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.497910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527c141-e87d-490f-a591-680b263c4a39 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.498315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527c141-e87d-490f-a591-680b263c4a39 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.498701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527c141-e87d-490f-a591-680b263c4a39 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.500200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 28/51] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:29.505406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.505713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.505975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.518258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82461193-8aee-4f01-b498-1efaee9f522c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.523690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:29.524907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:29.555408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.556294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.556576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.556847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.558053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:29.559558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:29.561312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 24/52] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:29 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:29.567038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.570792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:29.597663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 29/53] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:29.621044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.668773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd766b7-0f0b-4f6c-9720-dc68cfc4c81a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.689464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:29.704072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.704887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.705196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.705464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.706685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:29.708199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:29.709853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 25/54] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:29.716953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.725788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4845a-15e3-41fa-9345-4ffdcc763956 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.772773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.773584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.773890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.774284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.775479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:29.776971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:29.778829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 30/55] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:29.785013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:29.806079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:29.841623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b82c-27e4-4d6c-857b-383f1f1880ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.878085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.878741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.879067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.879337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.880184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.domain.id=default, user.id=73561edcff5e424d9ae49ab57406b79a) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:29.880490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:29.881929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb7b44-7145-46c1-8142-ce84461e574b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:29.890767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 26/56] 10.222.0.32 () {56 vars in 1322 bytes} [Wed May 25 14:25:29 2022] GET /identity/v3/role_assignments?user.id=73561edcff5e424d9ae49ab57406b79a&scope.domain.id=default&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 530 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:29.906098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee135-177f-4622-a51e-01628a12d8c4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:29.957304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d1a8d-1388-429b-9cfa-612943acb319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:29.957981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1a8d-1388-429b-9cfa-612943acb319 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:29.958269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1a8d-1388-429b-9cfa-612943acb319 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:29.958543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1a8d-1388-429b-9cfa-612943acb319 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:29.959323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d1a8d-1388-429b-9cfa-612943acb319 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:29.960845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d1a8d-1388-429b-9cfa-612943acb319 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:29.966054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:29.991126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 31/57] 10.222.0.32 () {60 vars in 1140 bytes} [Wed May 25 14:25:29 2022] POST /identity/v3/projects => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:25:32.234522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117786f-5baf-4250-acc5-d800b64843fb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:32.235448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117786f-5baf-4250-acc5-d800b64843fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:32.235448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117786f-5baf-4250-acc5-d800b64843fb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:32.237689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 27/58] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:32.245864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:32.246331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:32.246415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:32.265107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:32.266303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:32.337340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 32/59] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:32.362322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117786f-5baf-4250-acc5-d800b64843fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:32.463769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117786f-5baf-4250-acc5-d800b64843fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:32.498134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16286472-8ad4-4456-a79c-4251f8f4f767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:32.499120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286472-8ad4-4456-a79c-4251f8f4f767 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:32.499373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286472-8ad4-4456-a79c-4251f8f4f767 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:32.499687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16286472-8ad4-4456-a79c-4251f8f4f767 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:32.500963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16286472-8ad4-4456-a79c-4251f8f4f767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:32.502409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16286472-8ad4-4456-a79c-4251f8f4f767 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:32.504066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 28/60] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:32.509783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:32.612896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3604d3-5484-49b3-92dd-b8a8790af990 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:32.630547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:32.649967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585c191-3faa-4fd8-a8c6-1547703470b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:32.650653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585c191-3faa-4fd8-a8c6-1547703470b6 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:32.651041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585c191-3faa-4fd8-a8c6-1547703470b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:32.651354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585c191-3faa-4fd8-a8c6-1547703470b6 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:32.652092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585c191-3faa-4fd8-a8c6-1547703470b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:32.653569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585c191-3faa-4fd8-a8c6-1547703470b6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:32.680465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 33/61] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:25:32 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:25:36.049010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74045d-4740-434f-8334-1f1056974828 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.049910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74045d-4740-434f-8334-1f1056974828 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.050386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74045d-4740-434f-8334-1f1056974828 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.052082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 29/62] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:36.065749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041d436-56da-4e4a-bf0e-d27281133001 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.066767 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041d436-56da-4e4a-bf0e-d27281133001 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.067267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041d436-56da-4e4a-bf0e-d27281133001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.087993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d041d436-56da-4e4a-bf0e-d27281133001 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:36.089606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d041d436-56da-4e4a-bf0e-d27281133001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:36.181318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 34/63] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:36.204880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74045d-4740-434f-8334-1f1056974828 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:36.211925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:36.312998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b521842-da52-479d-bd93-eb26651307be None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.313289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b521842-da52-479d-bd93-eb26651307be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.313482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b521842-da52-479d-bd93-eb26651307be None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.314877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 35/64] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:36.320186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c2bc-36bb-4235-aa6b-d793e9ca3a42 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.320715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c2bc-36bb-4235-aa6b-d793e9ca3a42 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.321186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c2bc-36bb-4235-aa6b-d793e9ca3a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.350661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74045d-4740-434f-8334-1f1056974828 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:36.353035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7929c2bc-36bb-4235-aa6b-d793e9ca3a42 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:36.354278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7929c2bc-36bb-4235-aa6b-d793e9ca3a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:36.399130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5d6e5-f648-4649-a8c6-78fd77547ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:36.399984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5d6e5-f648-4649-a8c6-78fd77547ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.400258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5d6e5-f648-4649-a8c6-78fd77547ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.400525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5d6e5-f648-4649-a8c6-78fd77547ee2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.424313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5d6e5-f648-4649-a8c6-78fd77547ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-47e5d6e5-f648-4649-a8c6-78fd77547ee2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:36.433800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:36.434920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:36.435952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:36.438077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 30/65] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:36.439518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441222aa-cb99-4707-9629-bb22cff830a2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.440045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441222aa-cb99-4707-9629-bb22cff830a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.440454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441222aa-cb99-4707-9629-bb22cff830a2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.442003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 31/66] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:36.447035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441222aa-cb99-4707-9629-bb22cff830a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:36.467322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 36/67] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:36.469079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.469561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.470024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.486484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:36.488024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:36.525578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:36.534859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:36.576519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441222aa-cb99-4707-9629-bb22cff830a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:36.583371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 37/68] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:36.584604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:36.613223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:36.613906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.614184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.614447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.615236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:36.615537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:36.616928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b40003-2dc5-43ba-9a84-61e1ca392b0d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:36.625727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 32/69] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:36.627369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787c498-0021-4d28-9771-05b155b7fcfd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.627827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787c498-0021-4d28-9771-05b155b7fcfd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.628266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787c498-0021-4d28-9771-05b155b7fcfd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.629688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 33/70] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:36.630586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787c498-0021-4d28-9771-05b155b7fcfd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:36.698220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ab8ea-ba9d-4024-98d1-3a90ba2b3523 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:36.734314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb18609-1318-4f03-85fe-ef396de7ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:36.735168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb18609-1318-4f03-85fe-ef396de7ddfe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.735456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb18609-1318-4f03-85fe-ef396de7ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.735718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb18609-1318-4f03-85fe-ef396de7ddfe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.742376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787c498-0021-4d28-9771-05b155b7fcfd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:36.746711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb18609-1318-4f03-85fe-ef396de7ddfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7bb18609-1318-4f03-85fe-ef396de7ddfe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:36.754908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:36.755761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:36.756501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 38/71] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:36.757303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.757721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.758180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.774601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:36.774987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:36.776178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:36.787875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:36.788986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.789332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.789607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.799079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:36.803905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:36.811755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:36.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:36.813342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 34/72] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:36.813950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:36.848936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 39/73] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:36.850038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:36.928211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e32258-9b2b-4e43-bbaa-58aaaadc43cb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:36.964202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3bdaf-44bc-4baa-9aaf-94ea9a91ef44 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:36.971870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:36.972788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:36.973065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:36.973340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:36.988317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:36.989801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:36.991748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 35/74] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:36.992506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.000569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.001242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.001523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.001795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.002567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:37.002904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.004338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a5808-34df-433c-ac1e-d18f5aa6b46b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.012487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 40/75] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.013745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965a4be-b794-4623-afc8-d5fd4cd0a885 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.014056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965a4be-b794-4623-afc8-d5fd4cd0a885 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.014319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965a4be-b794-4623-afc8-d5fd4cd0a885 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.015607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 41/76] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:37.016281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965a4be-b794-4623-afc8-d5fd4cd0a885 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.035715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.043340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.100188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf97ad-c869-44da-933a-d8d28fdb64ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.122572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965a4be-b794-4623-afc8-d5fd4cd0a885 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.135847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.136530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.136815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.137091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.137851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:37.138155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.139597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.147595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 36/77] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.148377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.158713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.159560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.159841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.160117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.171240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:37.179371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:37.180183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:37.180901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 42/78] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:37.181510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.252001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8123933-df3e-43ed-b66a-29bc25590a2c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.257330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.259456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.286364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3532a-f38c-4661-a294-1bde4347bb07 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.290195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.290941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.291259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.291540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.320635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cc2d0-4b2f-487e-a7a5-98eb31768f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.321520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cc2d0-4b2f-487e-a7a5-98eb31768f6f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.321799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cc2d0-4b2f-487e-a7a5-98eb31768f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.322048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cc2d0-4b2f-487e-a7a5-98eb31768f6f None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.323272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cc2d0-4b2f-487e-a7a5-98eb31768f6f None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.324749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cc2d0-4b2f-487e-a7a5-98eb31768f6f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.326431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 43/79] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.327615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.327901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.328175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.344067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:37.345269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:37.363141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.364567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.413045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 44/80] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:37.414123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.473436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 37/81] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:37.474300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.482451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.496297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.522819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac377242-dbee-4a2c-8b58-e9c6996376db None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.557222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.558390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.558666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.558965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.560188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.561659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.563342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 45/82] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.564103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.578168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ba48d-fa5a-4e84-9c2e-1d69f2f372de None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.611129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.611793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.612075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.612349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.613082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:37.613438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.614776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.623382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 38/83] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.624284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.672095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258404c-4924-4eaf-a240-073924910f95 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.690048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.706798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.707455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.707735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.708007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.719767 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.723730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc971e-ac45-4900-9c3e-2a279a2a4bc2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.759508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.760326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.760608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.760884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.762115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.763607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.765336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 39/84] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.766120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.784331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.785871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.877940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf44103-f894-4faf-9ac6-28a006fb2919 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:37.892698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 46/85] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:37.893596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.916686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:37.917572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:37.917857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:37.918126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:37.919432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:37.920930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:37.922646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 40/86] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:37.923441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:37.928570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:37.943378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.005055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde59442-d75d-4e34-a81e-2868cff4ea1c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.035691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88ca13-7442-4f51-b670-7541c35954b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.041993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.042665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.042979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.043258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.077316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.078894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.079385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.079882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.081979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.084702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.087978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 41/87] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:38.089123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.116233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.117859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.161863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.163402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.199438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cdf38-5e4f-4367-940f-40a2dab39c1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.235949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.236631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.236916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.237254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.237908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.239393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.241615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 47/88] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:38.243157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.266148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 42/89] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:25:38 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:38.267387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.376973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00a61-81e7-43ae-a3fd-589a10b56868 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.381627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001112f-3c64-4724-9478-01c08f0b9526 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.414681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.417150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.418049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.418339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.420038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.421563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.423445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 48/90] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:38.428799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.431065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.432506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.434356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.434898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.436682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=a1a9cfcf6d5443678b5f66674068c1bf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:38.437305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.441462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.442784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.454014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 43/91] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/role_assignments?user.id=a1a9cfcf6d5443678b5f66674068c1bf&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=3e228f026e734b8b9c06f578f2012163 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:38.455728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.580334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2254ca-9a0f-485c-bfab-58a2171f7a5f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.617484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b77c6-0b53-496a-9eeb-8549354545d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.623842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.624595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.624889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.625168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.671139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.672116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.673279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.673279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.674873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.676458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.680460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 44/92] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:38.681413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.724533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.726960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.742326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.759199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:38.836367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db35f-d214-42a0-960c-63be563f7910 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:38.905869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:38.906549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:38.906854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:38.907131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:38.907984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] RBAC: Adding query filter params (role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=a1a9cfcf6d5443678b5f66674068c1bf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:38.908298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:38.909760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:38.915698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 49/93] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:38.916676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:38.925082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 45/94] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/role_assignments?user.id=a1a9cfcf6d5443678b5f66674068c1bf&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:38.926137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:39.118400 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e8e2d-9218-4476-8934-38341c0f67b1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:39.130916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d33e9b-c7f7-4abd-a3dc-0dce1da90b0a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:39.143171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:39.168643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:39.193282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:39.194103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:39.194387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:39.194660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] PATH_INFO: `/v3/projects/f58e62dd83ba4e83b88c10be4012cee8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:39.196130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] RBAC: Authorizing `identity:get_project(project_id=f58e62dd83ba4e83b88c10be4012cee8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:39.197629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:39.199460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 46/95] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/projects/f58e62dd83ba4e83b88c10be4012cee8 => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:39.205855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:39.223508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:39.224202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:39.224497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:39.224761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:39.225675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=73561edcff5e424d9ae49ab57406b79a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:39.225992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:39.227513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820107d5-1f11-4ba5-96e8-7f5eb1dc01c0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:39.245076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 50/96] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:38 2022] GET /identity/v3/role_assignments?user.id=73561edcff5e424d9ae49ab57406b79a&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:39.403194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf9a3-3e47-43a7-a381-fbb5cecc7a85 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:39.452785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:39.453395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:39.453686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:39.453963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:39.454864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=f58e62dd83ba4e83b88c10be4012cee8, user.id=a1a9cfcf6d5443678b5f66674068c1bf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:39.455184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:39.456710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1a05a-78c8-4328-8db2-a1f67aa6621e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:39.467611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 47/97] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:39 2022] GET /identity/v3/role_assignments?user.id=a1a9cfcf6d5443678b5f66674068c1bf&scope.project.id=f58e62dd83ba4e83b88c10be4012cee8&role.id=3e228f026e734b8b9c06f578f2012163 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:41.350704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffe148-b12a-4f37-90b9-e4fe6caf2244 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.351658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffe148-b12a-4f37-90b9-e4fe6caf2244 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.352084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffe148-b12a-4f37-90b9-e4fe6caf2244 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.353715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 51/98] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:41.359826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15344-38cb-40ee-8dd0-161fcf3987ed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.360790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15344-38cb-40ee-8dd0-161fcf3987ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.361246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b15344-38cb-40ee-8dd0-161fcf3987ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.387880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c8b15344-38cb-40ee-8dd0-161fcf3987ed None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:41.389455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8b15344-38cb-40ee-8dd0-161fcf3987ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:41.459850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:41.478068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 48/99] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:41.521433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffe148-b12a-4f37-90b9-e4fe6caf2244 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:41.545735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690b583-985c-4ecc-9d17-752eb91cfc09 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.546038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690b583-985c-4ecc-9d17-752eb91cfc09 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.546286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690b583-985c-4ecc-9d17-752eb91cfc09 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.547954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 49/100] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:41.554386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe200ae9-bba1-4065-8a5e-7ec999cfd570 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.554672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe200ae9-bba1-4065-8a5e-7ec999cfd570 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.554976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe200ae9-bba1-4065-8a5e-7ec999cfd570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.578877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fe200ae9-bba1-4065-8a5e-7ec999cfd570 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:41.580072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe200ae9-bba1-4065-8a5e-7ec999cfd570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:41.702139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffe148-b12a-4f37-90b9-e4fe6caf2244 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:41.705076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 50/101] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:41.706282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfdb0a-3d1f-42e6-9e2b-db6fbc032b59 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.706572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfdb0a-3d1f-42e6-9e2b-db6fbc032b59 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.706869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfdb0a-3d1f-42e6-9e2b-db6fbc032b59 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.708265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 51/102] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:41.716607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.717006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.717241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.742266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:41.743494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:41.774034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:41.774878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:41.775157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:41.775428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:41.777842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:41.782680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:41.782680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 52/103] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:41.782680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:41.867401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:41.877435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 52/104] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:41.878293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:41.959268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:41.967931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320b2ba-ce82-4520-a2de-80e6d384152f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.032044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.032873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.033152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.033426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.053301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:42.063226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:42.064108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:42.064925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:42.067764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36917993-e6e0-4a2f-a0d4-caa1c312bfd8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.069363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 53/105] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:42.071174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.131899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.134926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.135387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.135847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.136704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:42.143441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:42.197762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 53/106] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:25:41 2022] POST /identity/v3/projects => generated 302 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:42.199234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.297106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32858376-81e1-4c54-9d76-6d1cc5ff318b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.300511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:42.362567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1e774-5acc-469e-8dfe-f49f4383bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.363477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e774-5acc-469e-8dfe-f49f4383bf50 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.363768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e774-5acc-469e-8dfe-f49f4383bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.364105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e774-5acc-469e-8dfe-f49f4383bf50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.377501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1e774-5acc-469e-8dfe-f49f4383bf50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:42.387189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-17b1e774-5acc-469e-8dfe-f49f4383bf50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:42.390356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:42.391561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:42.392684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:42.393427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 54/107] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:42.395115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be704-d66b-4313-9cac-a5500be15134 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.395476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be704-d66b-4313-9cac-a5500be15134 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.395842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be704-d66b-4313-9cac-a5500be15134 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.397461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 55/108] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:42.398724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144fb68-b8e3-4a39-a295-ea2d0847dc93 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.399105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144fb68-b8e3-4a39-a295-ea2d0847dc93 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.399356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a69e-16b9-4d08-ab23-12c336cf51ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.399458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144fb68-b8e3-4a39-a295-ea2d0847dc93 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.400813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 56/109] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:42.401832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144fb68-b8e3-4a39-a295-ea2d0847dc93 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.449744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.450569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.451600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.451600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.452108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:42.452869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:42.453932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2642d-7540-4721-bf70-797e577d78f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:42.464652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 54/110] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:42.465951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cca8b-603c-46f2-b5e6-8d12c319d393 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.466248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cca8b-603c-46f2-b5e6-8d12c319d393 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.466854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cca8b-603c-46f2-b5e6-8d12c319d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.488737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b99cca8b-603c-46f2-b5e6-8d12c319d393 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:42.491160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b99cca8b-603c-46f2-b5e6-8d12c319d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:42.543662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144fb68-b8e3-4a39-a295-ea2d0847dc93 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.577198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 55/111] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:42.582822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.582822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.582822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.609504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:42.610798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:42.621542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.622596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.625235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.625498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.626301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:42.626627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:42.628131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:42.640521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 57/112] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:42.641445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.669531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:42.706051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 56/113] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:42.707554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.734915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:42.756301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656f7c4-7f18-4e02-a729-1d859f2fd9da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.801747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.803052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.803337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.803619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] PATH_INFO: `/v3/users/5fe26db3a2f942ca9f5017c3c05a773d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.819131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] RBAC: Authorizing `identity:get_user(user_id=5fe26db3a2f942ca9f5017c3c05a773d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:42.820810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:42.822913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 58/114] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/users/5fe26db3a2f942ca9f5017c3c05a773d => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:42.827988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.829304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e91c1-dfcc-4898-baf8-7131f2a696fc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.871305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.872669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.873234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.873707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.886148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:42.895340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:42.896175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:42.896905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:42.899924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 57/115] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:42.901117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.921019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:42.941510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3dc2-8425-4194-b9e0-a68e3a4e9e39 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:42.980685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:42.981620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:42.981906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:42.982697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:42.983645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:42.985112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:42.987209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 59/116] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:42.987767 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:42.999973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.022123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0ed8-7f61-4fb0-bcf0-a5eb72eb5ef8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.061219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.063092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.063607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.064300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.078411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:43.087557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:43.088559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:43.089423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 58/117] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:43.090179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.124732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdd275-5226-4d65-959c-660d76bf8a78 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.165353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaa3fe-526e-4417-b1f2-ae99e8b43560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.166116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaa3fe-526e-4417-b1f2-ae99e8b43560 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.166421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaa3fe-526e-4417-b1f2-ae99e8b43560 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.166696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaa3fe-526e-4417-b1f2-ae99e8b43560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.188820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.204786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ca3ec-f8f0-47ed-b948-80669b0c8b62 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.241782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.242402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaa3fe-526e-4417-b1f2-ae99e8b43560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.242641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.243016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.243362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.244217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaa3fe-526e-4417-b1f2-ae99e8b43560 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.244311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:43.244633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.246068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.254969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 59/118] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:43.255578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.258109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.356421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 60/119] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:43.357845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73930d3-1681-4bcc-804d-114018c7444d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.358136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73930d3-1681-4bcc-804d-114018c7444d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.358423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73930d3-1681-4bcc-804d-114018c7444d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.359919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 61/120] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:43.362600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6e270-bbd7-4c34-80d8-6e56aaa84ecb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.364462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73930d3-1681-4bcc-804d-114018c7444d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.397215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.398091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.398378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.398681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.399929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.401375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.403296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 60/121] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:43.408583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.420115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.473275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73930d3-1681-4bcc-804d-114018c7444d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.483352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.511091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.511780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.512053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.512324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.513082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:43.513385 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.514859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.519947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc4b70-0d58-4e07-a54b-e860a1f5c14b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.524726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 62/122] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:43.525700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.571369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871d052-b7bd-448d-9298-44cc5bcfb7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.573178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871d052-b7bd-448d-9298-44cc5bcfb7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.573463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871d052-b7bd-448d-9298-44cc5bcfb7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.574045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871d052-b7bd-448d-9298-44cc5bcfb7b5 None admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.575250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871d052-b7bd-448d-9298-44cc5bcfb7b5 None admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.576853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871d052-b7bd-448d-9298-44cc5bcfb7b5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.581182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 61/123] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:43.584182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.585827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.586275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.605964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:43.607602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:43.661781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2b9da-be93-4214-b8d3-628ae213876d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.682390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.713591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 62/124] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:43.720272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.721930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.722520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.723133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.723966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.725580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.728816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.738787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 63/125] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:43.738787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.824923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:43.866854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47673599-a425-47b9-ac7d-01335f1c11ee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.883694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f03f7f-0a7f-49dd-9c44-e8c09125a987 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:43.909710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.910627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.911000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.911276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/a1a9cfcf6d5443678b5f66674068c1bf/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.913340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] RBAC: Authorizing `identity:create_grant(project_id=141ee3a6fe3b41d5a050afe46844c163, user_id=a1a9cfcf6d5443678b5f66674068c1bf, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.915058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.919957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:43.921113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:43.921402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:43.921740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:43.923146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:43.924710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:43.931307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 64/126] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:43.931307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.937612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 63/127] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:43 2022] PUT /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/a1a9cfcf6d5443678b5f66674068c1bf/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:43.951799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:43.991099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:44.077966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1900-463d-4c9e-b560-86d24406a084 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.091972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15cf4-1453-4455-bab5-22d954efd69d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.120303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.121123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.121401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.121673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.123218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:44.125394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:44.131361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 65/128] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:44.131361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:44.140897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:44.165938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.166974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.167119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.167405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.168266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] RBAC: Adding query filter params (role.id=9b576d9657c34867877c2d8958bfa180, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=5fe26db3a2f942ca9f5017c3c05a773d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:44.168579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:44.170101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:44.183615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 64/129] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:43 2022] GET /identity/v3/role_assignments?user.id=5fe26db3a2f942ca9f5017c3c05a773d&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=9b576d9657c34867877c2d8958bfa180 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:44.184884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:44.316144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e1a2a-fc4b-44d9-9d09-9d8dbf5cc5b3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.338829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:44.364340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.365624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.368122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.368715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.383573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:44.392540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:44.393522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:44.394295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:44.397482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 66/130] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:44.399066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:44.473745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9497838-c950-4f21-8017-370bcbd56ebf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.521102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.521954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.522232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.522503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.523783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:44.525251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:44.527376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 65/131] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:44 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:44.528079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2375b7-9574-49ef-bbb0-a2b4798952d1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.528209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:44.569645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.571033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.571910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.572401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/73561edcff5e424d9ae49ab57406b79a/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.574863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] RBAC: Authorizing `identity:create_grant(project_id=141ee3a6fe3b41d5a050afe46844c163, user_id=73561edcff5e424d9ae49ab57406b79a, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:44.576939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:44.597599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 67/132] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:44 2022] PUT /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/73561edcff5e424d9ae49ab57406b79a/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:44.599059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:44.657260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:44.678984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:44.768023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55d5c-cdbd-43d9-8e85-d5a6cbac6ecb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.769938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a677-1584-4cdc-9620-c3b3a79019f2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:44.850169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9754e-c811-4fa6-ac04-8ea0d5932041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.851150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9754e-c811-4fa6-ac04-8ea0d5932041 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.851431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9754e-c811-4fa6-ac04-8ea0d5932041 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.851704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9754e-c811-4fa6-ac04-8ea0d5932041 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/a1a9cfcf6d5443678b5f66674068c1bf/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.853052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:44.853739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.854014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.854284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.855091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:44.856098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:44.857618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:44.860678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9754e-c811-4fa6-ac04-8ea0d5932041 None admin] RBAC: Authorizing `identity:create_grant(project_id=141ee3a6fe3b41d5a050afe46844c163, user_id=a1a9cfcf6d5443678b5f66674068c1bf, role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:44.862183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9754e-c811-4fa6-ac04-8ea0d5932041 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:44.878876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 66/133] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:44.885331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:44.899441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 68/134] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:44 2022] PUT /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/a1a9cfcf6d5443678b5f66674068c1bf/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:44.908742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73f9f-8aa9-4946-8954-cc9ea5d67c9d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.908918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73f9f-8aa9-4946-8954-cc9ea5d67c9d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.909206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d73f9f-8aa9-4946-8954-cc9ea5d67c9d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.910764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 69/135] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:25:44.916807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:44.917143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:44.917369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:44.944010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:44.945267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:45.075649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39a660-0a59-47d6-8dd5-d1158022cffe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:45.104221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 70/136] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:45.129847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:45.147795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:45.148703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:45.148983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:45.149256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:45.150502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:45.152091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:45.153738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 67/137] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:44 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:45.157790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:45.161283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:45.210085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:45.283576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ed601-0601-4b23-8ae3-e5608f0a52f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:45.322990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:45.323936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:45.324150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:45.324420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] PATH_INFO: `/v3/projects/f58e62dd83ba4e83b88c10be4012cee8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:45.325634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] RBAC: Authorizing `identity:get_project(project_id=f58e62dd83ba4e83b88c10be4012cee8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:45.327404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:45.329481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 68/138] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:45 2022] GET /identity/v3/projects/f58e62dd83ba4e83b88c10be4012cee8 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:45.330906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbfacf-51da-49e4-8656-eca92d86fc82 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:45.335280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:45.369611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:45.371337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:45.371818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:45.372357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:45.374082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:45.375864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:45.377847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 71/139] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:45.384532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:45.418145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:45.471915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d3f62-3f37-433f-b544-7c66ba447c5b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:45.509033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56944afa-9444-42a7-b0cf-3ba461e38dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:45.509964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56944afa-9444-42a7-b0cf-3ba461e38dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:45.510310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56944afa-9444-42a7-b0cf-3ba461e38dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:45.510651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56944afa-9444-42a7-b0cf-3ba461e38dc0 None admin] PATH_INFO: `/v3/projects/f58e62dd83ba4e83b88c10be4012cee8/users/a1a9cfcf6d5443678b5f66674068c1bf/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:45.512811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56944afa-9444-42a7-b0cf-3ba461e38dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f58e62dd83ba4e83b88c10be4012cee8, user_id=a1a9cfcf6d5443678b5f66674068c1bf, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:45.514553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56944afa-9444-42a7-b0cf-3ba461e38dc0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:45.525485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:45.534296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491a86-7590-42df-8739-fc55cbf23306 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:45.534466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 69/140] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:45 2022] PUT /identity/v3/projects/f58e62dd83ba4e83b88c10be4012cee8/users/a1a9cfcf6d5443678b5f66674068c1bf/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:25:45.581678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1bc1-4d4b-43da-87c1-d7dca882cb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:45.582822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1bc1-4d4b-43da-87c1-d7dca882cb2f None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:45.583298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1bc1-4d4b-43da-87c1-d7dca882cb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:45.583759 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1bc1-4d4b-43da-87c1-d7dca882cb2f None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:45.584764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1bc1-4d4b-43da-87c1-d7dca882cb2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:45.586564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1bc1-4d4b-43da-87c1-d7dca882cb2f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:45.630778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 72/141] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:25:45 2022] POST /identity/v3/users => generated 283 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:25:46.826443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80480ce6-ac58-4043-97bf-05d28b51acd9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:46.826981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80480ce6-ac58-4043-97bf-05d28b51acd9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:46.827081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80480ce6-ac58-4043-97bf-05d28b51acd9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:46.828404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 70/142] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:46.835238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:46.836171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:46.836647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:46.858569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:46.860126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:46.953806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 73/143] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:46.981912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80480ce6-ac58-4043-97bf-05d28b51acd9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:47.127540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80480ce6-ac58-4043-97bf-05d28b51acd9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:47.155191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:47.175383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde22362-a933-4db6-82a3-b2c72f5a28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:47.176152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde22362-a933-4db6-82a3-b2c72f5a28f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.176497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde22362-a933-4db6-82a3-b2c72f5a28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.176773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde22362-a933-4db6-82a3-b2c72f5a28f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:47.192526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde22362-a933-4db6-82a3-b2c72f5a28f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cde22362-a933-4db6-82a3-b2c72f5a28f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:47.203595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:47.204495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:47.205697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 71/144] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:47.211003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:47.314645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:47.336101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0747-28ad-4a72-9e86-268ea3f84381 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.336782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0747-28ad-4a72-9e86-268ea3f84381 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.337575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c0747-28ad-4a72-9e86-268ea3f84381 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:47.339623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 72/145] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:47.342837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4e589-9ccb-488d-8833-c52bbb8995d4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:47.346477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.347027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.347533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:47.377342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:47.379066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:47.389182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:47.389868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.390155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.390431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:47.391236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:47.391545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:47.393091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:47.408736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 74/146] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:47.410950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:47.472043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 73/147] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:47.513554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:47.581605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf802e62-cb4a-447d-af75-2b908e4d03e5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:47.641456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:47.642820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.643349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.643826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:47.646607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:47.648662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:47.651181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 75/148] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:47 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:47.656723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:47.679736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:47.741712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a1b56-3926-4a39-80f2-063f0d4fe4a8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:47.758198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:47.802223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:47.803084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.803369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.803641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:47.825134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:47.838919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185bba7-8484-4a0a-ae6c-5a8898930cf0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:47.844707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:47.845820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:47.846561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 74/149] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:47.857280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:47.912672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf72fed-e6f5-483a-b0d5-d296c5b3b6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:47.913267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf72fed-e6f5-483a-b0d5-d296c5b3b6be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:47.913607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf72fed-e6f5-483a-b0d5-d296c5b3b6be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:47.913960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf72fed-e6f5-483a-b0d5-d296c5b3b6be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.002879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c0cac-02f1-40d4-bdf8-94a46d28e7df None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:48.050972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf72fed-e6f5-483a-b0d5-d296c5b3b6be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:48.052980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf72fed-e6f5-483a-b0d5-d296c5b3b6be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:48.068128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:48.068823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.069101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.069375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.070593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:48.071167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:48.073738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:48.074982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:48.088186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 75/150] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:48.094387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:48.148617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:48.256300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 76/151] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:25:48.257811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cbf20-109b-44c8-85b8-27443f038ad0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.258104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cbf20-109b-44c8-85b8-27443f038ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.258380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2cbf20-109b-44c8-85b8-27443f038ad0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.259800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 77/152] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:48.264303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cbf20-109b-44c8-85b8-27443f038ad0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:48.278658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e568f-d211-4482-966f-9122b90ca3d7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:48.347406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d08d1-fddb-4fa8-b712-46bf96b7599f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:48.348653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d08d1-fddb-4fa8-b712-46bf96b7599f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.349123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d08d1-fddb-4fa8-b712-46bf96b7599f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.350608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d08d1-fddb-4fa8-b712-46bf96b7599f None admin] PATH_INFO: `/v3/users/5fe26db3a2f942ca9f5017c3c05a773d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.353689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d08d1-fddb-4fa8-b712-46bf96b7599f None admin] RBAC: Authorizing `identity:get_user(user_id=5fe26db3a2f942ca9f5017c3c05a773d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:48.355414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d08d1-fddb-4fa8-b712-46bf96b7599f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:48.357914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 76/153] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:48 2022] GET /identity/v3/users/5fe26db3a2f942ca9f5017c3c05a773d => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:48.359319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.360016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.360497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.388768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:48.390333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:48.473674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2cbf20-109b-44c8-85b8-27443f038ad0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:48.492790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:48.493333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 77/154] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:48.494267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:48.530504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:48.542304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:48.543263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.543559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.543825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.545131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:48.546876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:48.548847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 78/155] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:48 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:48.551527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:48.645985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74964f-6eca-4cab-ba02-f4ec52b7746f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:48.724606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:48.725819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.726283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.726735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.728189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:48.729832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:48.735939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 78/156] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:48 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:48.736905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c6d59-9368-4609-bfb8-67f371a83883 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:48.737152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:48.793436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:48.794178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:48.794458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:48.794822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:48.816163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:48.828334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:48.829274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:48.830029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 79/157] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:48.830693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:48.913901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:48.976105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:48.989369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db113eb-8f02-4ba0-b6cc-da2008ae0dbe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:49.013100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc383f-a697-427d-b17d-869d9d4a20de None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:49.079353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:49.080137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.080500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.080771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.081627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=a1a9cfcf6d5443678b5f66674068c1bf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:49.081933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:49.084029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:49.102566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 79/158] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:48 2022] GET /identity/v3/role_assignments?user.id=a1a9cfcf6d5443678b5f66674068c1bf&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=3e228f026e734b8b9c06f578f2012163 => generated 608 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:49.103634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:49.104897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:49.106673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.107340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.107825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/5fe26db3a2f942ca9f5017c3c05a773d/roles/9b576d9657c34867877c2d8958bfa180` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.117758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] RBAC: Authorizing `identity:create_grant(project_id=141ee3a6fe3b41d5a050afe46844c163, user_id=5fe26db3a2f942ca9f5017c3c05a773d, role_id=9b576d9657c34867877c2d8958bfa180)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:49.119803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:49.148220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 80/159] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:48 2022] PUT /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163/users/5fe26db3a2f942ca9f5017c3c05a773d/roles/9b576d9657c34867877c2d8958bfa180 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:49.290444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c47bd-1ab6-49cb-a9cf-016499794873 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:49.344211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:49.344987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.345269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.345538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.346331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:49.346645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:49.348178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:49.358604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 80/160] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:49.364837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:49.508779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:49.512645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dab4e-db78-4f17-9e70-34f5ea2d908e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:49.558208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af772e-8896-4526-b3b0-54da4543148b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:49.559147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af772e-8896-4526-b3b0-54da4543148b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.559428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af772e-8896-4526-b3b0-54da4543148b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.559694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af772e-8896-4526-b3b0-54da4543148b None admin] PATH_INFO: `/v3/users/73561edcff5e424d9ae49ab57406b79a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.561033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af772e-8896-4526-b3b0-54da4543148b None admin] RBAC: Authorizing `identity:get_user(user_id=73561edcff5e424d9ae49ab57406b79a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:49.562511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af772e-8896-4526-b3b0-54da4543148b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:49.564527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 81/161] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:49 2022] GET /identity/v3/users/73561edcff5e424d9ae49ab57406b79a => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:49.569912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:49.614264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:49.719867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871046a-c529-4c4e-9a1b-efb1d67a1345 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:49.760259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:49.760962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.761243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.761513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.868027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:49.869429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:49.890190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b2c30-fd96-4e58-b4b2-14f2425846e8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.890523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b2c30-fd96-4e58-b4b2-14f2425846e8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.890739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b2c30-fd96-4e58-b4b2-14f2425846e8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.892220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 82/162] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:49.894866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:49.899771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d1469-72d8-4b5f-b567-7fcba1094e3d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:49.899983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d1469-72d8-4b5f-b567-7fcba1094e3d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:49.900247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d1469-72d8-4b5f-b567-7fcba1094e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:49.942901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-346d1469-72d8-4b5f-b567-7fcba1094e3d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:49.944128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-346d1469-72d8-4b5f-b567-7fcba1094e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:50.066042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 81/163] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:50.067219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:50.149171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 83/164] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:50.184428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29ba18-1b93-4c45-bab0-cdb110084193 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:50.194705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8739aa-9107-4feb-805d-53c82d9c1e39 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:50.195159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8739aa-9107-4feb-805d-53c82d9c1e39 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:50.195449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8739aa-9107-4feb-805d-53c82d9c1e39 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:50.197172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 84/165] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:50.197970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8739aa-9107-4feb-805d-53c82d9c1e39 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:50.212943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:50.233893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112830ab-2800-4d50-8f68-55890078382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:50.234842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112830ab-2800-4d50-8f68-55890078382d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:50.235122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112830ab-2800-4d50-8f68-55890078382d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:50.235445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112830ab-2800-4d50-8f68-55890078382d None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:50.236717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112830ab-2800-4d50-8f68-55890078382d None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:50.238235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112830ab-2800-4d50-8f68-55890078382d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:50.240024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 82/166] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:50 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:50.241204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:50.241491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:50.241840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:50.272025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:50.273238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:50.393364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 83/167] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:50.394220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:50.440028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:50.455907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8739aa-9107-4feb-805d-53c82d9c1e39 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:50.492417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:50.493241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:50.493534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:50.493791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:50.505179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:50.513718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:50.514829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:50.516044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 85/168] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:25:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:50.516684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:50.603056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19c1e6-2a29-4775-968f-2ae33a90d4ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:50.627155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:50.670491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:50.671202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:50.671647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:50.671931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:50.672794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=73561edcff5e424d9ae49ab57406b79a) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:50.673104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:50.674567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:50.688668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 84/169] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:50 2022] GET /identity/v3/role_assignments?user.id=73561edcff5e424d9ae49ab57406b79a&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:50.689632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ab469-ef08-4203-b938-948a4f018574 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:50.692210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:50.772455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:50.773346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:50.773625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:50.773894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:50.807474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:50.819545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:50.820479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:50.821234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 86/170] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:50.831885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:50.875974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:50.964971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c8682-d07a-486e-af7a-f67c207c76d4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.047637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.048734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.049208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.049734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.049908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f7132-0690-4a1b-8e2d-42ab1db48148 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.050851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:51.051352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:51.053271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:51.066048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 85/171] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:25:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:51.079811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:51.123439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.124156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.124442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.124723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.125522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:51.126021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:51.127941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:51.145051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 87/172] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:51.151376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:51.154099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:51.237804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4a9b5-3bd7-443d-9748-1791803704f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.323465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c189b63-e37b-40b5-a19f-0fe74051cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.324006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:51.325865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c189b63-e37b-40b5-a19f-0fe74051cda6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.327273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c189b63-e37b-40b5-a19f-0fe74051cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.329397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c189b63-e37b-40b5-a19f-0fe74051cda6 None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.331586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c189b63-e37b-40b5-a19f-0fe74051cda6 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:51.334121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c189b63-e37b-40b5-a19f-0fe74051cda6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:51.336922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 86/173] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:51 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:51.343330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680793f-e2c5-432d-98b9-5fa2bc9a0189 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.343797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680793f-e2c5-432d-98b9-5fa2bc9a0189 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.344215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680793f-e2c5-432d-98b9-5fa2bc9a0189 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.345699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 87/174] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:51.346533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680793f-e2c5-432d-98b9-5fa2bc9a0189 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:51.377694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44b2d0-96ba-44d0-adf8-c1285ad8d6fa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.432725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec571924-1315-4e65-8900-c5affe2edb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.433898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571924-1315-4e65-8900-c5affe2edb03 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.434317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571924-1315-4e65-8900-c5affe2edb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.434733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec571924-1315-4e65-8900-c5affe2edb03 None admin] PATH_INFO: `/v3/users/a1a9cfcf6d5443678b5f66674068c1bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.437344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec571924-1315-4e65-8900-c5affe2edb03 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a9cfcf6d5443678b5f66674068c1bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:51.439549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec571924-1315-4e65-8900-c5affe2edb03 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:51.443556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 88/175] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:51 2022] GET /identity/v3/users/a1a9cfcf6d5443678b5f66674068c1bf => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:51.449542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.450042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.450477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.485646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:51.488121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:51.535047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680793f-e2c5-432d-98b9-5fa2bc9a0189 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.550421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:51.574051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f5604-001e-4be8-bcf6-7880d9f46b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.574706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f5604-001e-4be8-bcf6-7880d9f46b8f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.575053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f5604-001e-4be8-bcf6-7880d9f46b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.575289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f5604-001e-4be8-bcf6-7880d9f46b8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.577037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 89/176] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:51.577775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:51.653312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f5604-001e-4be8-bcf6-7880d9f46b8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:51.654905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f5604-001e-4be8-bcf6-7880d9f46b8f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:51.655800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:51.681052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a3254-3315-4372-bba5-33612c993042 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.714243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc062d-ab37-4016-bada-208e8fb5cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.714928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc062d-ab37-4016-bada-208e8fb5cb4b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.715220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc062d-ab37-4016-bada-208e8fb5cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.715468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc062d-ab37-4016-bada-208e8fb5cb4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.768242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 88/177] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:51.769908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d22f9-2544-4da8-8114-b154d06142af None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.770389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d22f9-2544-4da8-8114-b154d06142af None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.770879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d22f9-2544-4da8-8114-b154d06142af None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.772345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 89/178] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:51.773604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd786d-5495-42b9-9513-6df5d902584e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.774078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd786d-5495-42b9-9513-6df5d902584e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.774551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd786d-5495-42b9-9513-6df5d902584e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.775966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 90/179] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:51.780820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd786d-5495-42b9-9513-6df5d902584e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:51.797553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:51.800377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc062d-ab37-4016-bada-208e8fb5cb4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:51.802300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc062d-ab37-4016-bada-208e8fb5cb4b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:51.881583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:51.897512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd786d-5495-42b9-9513-6df5d902584e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:51.908568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 90/180] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:51.910121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47c240-f65e-4566-84d2-092744cc00b3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.910653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47c240-f65e-4566-84d2-092744cc00b3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.911078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47c240-f65e-4566-84d2-092744cc00b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.928617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4e47c240-f65e-4566-84d2-092744cc00b3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:51.929814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4e47c240-f65e-4566-84d2-092744cc00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:51.939913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:51.940720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:51.941006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:51.941265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:51.952438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:51.960087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:51.961162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 91/181] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:51.967458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.007459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 91/182] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:52.008792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.009150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.009536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.026394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:52.027644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:52.064075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.090517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f8ee-204d-47c4-9e2b-792767ed1d04 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.101021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 92/183] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:52.101671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.137602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.143536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.144406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.144683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.144950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.146139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.147618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.149770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 92/184] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:51 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.150520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.213459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01e0a6-9853-489d-a92d-6ae1b4db8c91 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.259085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.260083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.260364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.260636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] PATH_INFO: `/v3/projects/f58e62dd83ba4e83b88c10be4012cee8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.261974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] RBAC: Authorizing `identity:get_project(project_id=f58e62dd83ba4e83b88c10be4012cee8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.263528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.265304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 93/185] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/projects/f58e62dd83ba4e83b88c10be4012cee8 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.266893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.274062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61f0f0-c628-41e3-86b3-f33a0f15312d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.329992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.330587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.330939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.331217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.332010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:52.332303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.333734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.349227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 93/186] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.349227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.356753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.396786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72e334-354f-45ea-9f0f-f997bce3cc45 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.400599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.442346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.443305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.443586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.443859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.456008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:52.464423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:52.465424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:52.466211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 94/187] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:52.468021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.494213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02f1b5-0b01-4b30-96d2-84ba862c6a29 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.530355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.531555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.531975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.532386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.534189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.536505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.539188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 94/188] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.542856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.588594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9023-569b-41ca-9c32-4439761ae8e0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.630110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.630779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.631086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.631368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.632191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] RBAC: Adding query filter params (role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=a1a9cfcf6d5443678b5f66674068c1bf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:52.632502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.633984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.637435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.640834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 95/189] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/role_assignments?user.id=a1a9cfcf6d5443678b5f66674068c1bf&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.644252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.673081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78119b-a344-4d61-89b1-6f3f2495d164 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.687307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.714478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.715201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.715481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.715750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.716680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=f58e62dd83ba4e83b88c10be4012cee8, user.id=a1a9cfcf6d5443678b5f66674068c1bf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:52.716987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.718721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.726295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 95/190] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/role_assignments?user.id=a1a9cfcf6d5443678b5f66674068c1bf&scope.project.id=f58e62dd83ba4e83b88c10be4012cee8&role.id=3e228f026e734b8b9c06f578f2012163 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.727243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.796391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418169b-a4dc-412b-b4aa-3976cb07c9e0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.848652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b971725-bd7e-49a6-b03c-5bab7ea9e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.849903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b971725-bd7e-49a6-b03c-5bab7ea9e701 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.850284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b971725-bd7e-49a6-b03c-5bab7ea9e701 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.850657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b971725-bd7e-49a6-b03c-5bab7ea9e701 None admin] PATH_INFO: `/v3/users/87aee27909464137b36ff8cae338ae46` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.872044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b971725-bd7e-49a6-b03c-5bab7ea9e701 None admin] RBAC: Authorizing `identity:get_user(user_id=87aee27909464137b36ff8cae338ae46)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.874287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b971725-bd7e-49a6-b03c-5bab7ea9e701 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.876511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 96/191] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/users/87aee27909464137b36ff8cae338ae46 => generated 283 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.878338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc65b-ffc7-4f38-863e-5250b9c2c141 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.878848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc65b-ffc7-4f38-863e-5250b9c2c141 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.879332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfc65b-ffc7-4f38-863e-5250b9c2c141 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.881068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 97/192] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:52.882101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc65b-ffc7-4f38-863e-5250b9c2c141 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:52.891926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588ce2e-2d04-4147-8916-4fcd35d28926 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:52.933703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:52.933940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:52.934183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:52.934341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:52.938162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:52.938162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:52.938162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:52.939866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:52.949204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 96/193] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:52.950248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.019193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.033718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfc65b-ffc7-4f38-863e-5250b9c2c141 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.069816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6f993-702b-4642-8d8a-61eb482b3c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.070492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f993-702b-4642-8d8a-61eb482b3c38 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.070770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f993-702b-4642-8d8a-61eb482b3c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.071057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f993-702b-4642-8d8a-61eb482b3c38 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.071826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6f993-702b-4642-8d8a-61eb482b3c38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.073263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6f993-702b-4642-8d8a-61eb482b3c38 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:53.082830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c582-c7ff-41eb-a7f0-f9d9a56b48eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.098196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 98/194] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:25:52 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:25:53.104501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.105207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.105207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.125129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:53.126465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:53.128420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.129101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.129377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.129645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.222608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 99/195] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:53.223368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.236094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.242877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.244620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:53.306110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.345533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf826a9-9677-4d94-9c6f-ab246f1d3570 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.365276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 97/196] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:53.366400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.402540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.403548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.403849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.404130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] PATH_INFO: `/v3/users/87aee27909464137b36ff8cae338ae46` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.405619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] RBAC: Authorizing `identity:get_user(user_id=87aee27909464137b36ff8cae338ae46)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.406993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:53.409034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 100/197] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/users/87aee27909464137b36ff8cae338ae46 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:53.409891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.508290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.512517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b7661-2e49-4f63-ba0d-15240e929fca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.541369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef0be3-1a62-49f2-bddd-b251d1dbe4a5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.555325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.556559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.556950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.557505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.572651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:53.582975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:53.583853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:53.584605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:53.585168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.585168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.585168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.585358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 98/198] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:53.585430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.585495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.586362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.587512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:53.590421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 101/199] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:53.596362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.611997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.722738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3640c-a937-4f31-821e-343816cb2dfb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.745335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8098c-caff-4b5d-94b0-806b03b7b621 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.768862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.769857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.770150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.770419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.790127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.791146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.791546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.791984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.793152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:53.793536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.795053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:53.808752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 102/200] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:53.810844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.817642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.869609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.871348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:53.890048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:53.930241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a125d42-04af-4e31-89f7-1912589abb98 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:53.978566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:53.979303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:53.979652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:53.979995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:53.980347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 99/201] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:53.981475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:53.986866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=87aee27909464137b36ff8cae338ae46) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:53.997130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:53.999042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.005536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 103/202] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/role_assignments?user.id=87aee27909464137b36ff8cae338ae46&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:54.011975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:54.051296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:54.098286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e097e-48df-480b-b653-5334356ce665 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:54.136471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:54.138145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:54.138677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:54.139316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] PATH_INFO: `/v3/users/5fe26db3a2f942ca9f5017c3c05a773d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:54.140878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] RBAC: Authorizing `identity:get_user(user_id=5fe26db3a2f942ca9f5017c3c05a773d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:54.142583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.144892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 100/203] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:53 2022] GET /identity/v3/users/5fe26db3a2f942ca9f5017c3c05a773d => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:54.151133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:54.221004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070bb7a-d12d-4924-b33c-dce67630d8c7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:54.282483 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf2d96-9e79-4ff1-8161-716ae4be2964 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:54.304253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:54.305145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:54.305424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:54.305694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:54.306933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:54.308528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.310571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 104/204] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:54 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:54.313741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:54.316389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:54.319362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:54.324998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:54.325979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:54.326475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:54.327000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:54.399359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:54.401205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.447375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedc637-8ea7-468a-813f-9dc3e4c1bed6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:54.484809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:54.484809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:54.485086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:54.485189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:54.486029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] RBAC: Adding query filter params (role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=87aee27909464137b36ff8cae338ae46) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:54.486353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:54.487932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.495430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 105/205] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:54 2022] GET /identity/v3/role_assignments?user.id=87aee27909464137b36ff8cae338ae46&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:54.529097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 101/206] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:54.535249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:54.639404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:54.679508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725fbe1-34cf-4621-9902-4c8334e7313d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:54.718209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42b980-7cf6-4d26-a571-915d81c03a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:54.719586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42b980-7cf6-4d26-a571-915d81c03a8a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:54.720006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42b980-7cf6-4d26-a571-915d81c03a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:54.720450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42b980-7cf6-4d26-a571-915d81c03a8a None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:54.722280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42b980-7cf6-4d26-a571-915d81c03a8a None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:54.724429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42b980-7cf6-4d26-a571-915d81c03a8a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.726859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 106/207] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:54 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:54.733233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:54.784826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:54.876437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48834047-f740-4698-a5a9-57c327e64cee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:54.918083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:54.918780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:54.919072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:54.919349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:54.920272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] RBAC: Adding query filter params (role.id=9b576d9657c34867877c2d8958bfa180, scope.project.id=141ee3a6fe3b41d5a050afe46844c163, user.id=5fe26db3a2f942ca9f5017c3c05a773d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:54.920585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:54.922166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b130e4-d93c-4165-971d-f9e2675384e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:54.929261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 102/208] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:54 2022] GET /identity/v3/role_assignments?user.id=5fe26db3a2f942ca9f5017c3c05a773d&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=9b576d9657c34867877c2d8958bfa180 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:55.853044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a96cd-b432-430b-9854-ba110014079a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:55.854070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a96cd-b432-430b-9854-ba110014079a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:55.854541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a96cd-b432-430b-9854-ba110014079a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:55.856773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 107/209] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:55.863499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2f08-e6cf-4c46-9e18-420f8a78a8b5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:55.864055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2f08-e6cf-4c46-9e18-420f8a78a8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:55.864230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:55.864679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:55.864752 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:55.865208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2f08-e6cf-4c46-9e18-420f8a78a8b5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:55.867049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 108/210] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:55.873717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:55.874291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:55.874782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:55.891230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:55.892694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:55.892981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:55.894358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:55.970407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 109/211] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:55.973143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 103/212] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:55.995684 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:55.998391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:56.023094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:56.074327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:56.128863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022787-f17a-4ee8-95c3-cc627dffb6f5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.148230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a368010-dd75-4f5e-8cce-571d58fc1c2f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.170787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.171705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.171983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.172258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.173525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:56.175133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:56.177415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 110/213] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:25:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:56.182661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:56.201593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.203090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.204069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.204594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.223259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:56.238626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:56.239566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:25:56.240333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:56.241521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 104/214] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:56.250210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:56.308272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd2856-8152-4464-9bf8-bc19805f6ba2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.339982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:56.345491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.346180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.346458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.346734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.347481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:56.349063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:56.370194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:56.375410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 111/215] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:25:56 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:25:56.395407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83e8bd-1c54-42c4-b410-86413604c3c9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.437000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.438150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.438648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.439184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.440276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:56.440817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:56.442476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:56.454310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 105/216] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:56.460211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:56.584397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d7f12-c045-4bc8-bf39-13e53d7d61de None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.624655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88b2fa-7fef-45cd-970d-89ec978d00fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.625469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88b2fa-7fef-45cd-970d-89ec978d00fc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.625747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88b2fa-7fef-45cd-970d-89ec978d00fc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.626020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88b2fa-7fef-45cd-970d-89ec978d00fc None admin] PATH_INFO: `/v3/users/935ef8de15954dbeb2c80cd90e444b0e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.640557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88b2fa-7fef-45cd-970d-89ec978d00fc None admin] RBAC: Authorizing `identity:get_user(user_id=935ef8de15954dbeb2c80cd90e444b0e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:56.642050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88b2fa-7fef-45cd-970d-89ec978d00fc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:56.644136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 112/217] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:56 2022] GET /identity/v3/users/935ef8de15954dbeb2c80cd90e444b0e => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:56.649668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:56.655208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fd1ce-1ceb-42dd-82bc-3f90e79af3ef None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.655491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fd1ce-1ceb-42dd-82bc-3f90e79af3ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.655766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fd1ce-1ceb-42dd-82bc-3f90e79af3ef None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.657322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 113/218] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:25:56.664315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.664598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.664867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.682586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:56.683873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:56.739031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:56.774618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 114/219] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:25:56.796183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe150d-5a67-480f-87f5-509f9c06cb3e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.798140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:56.837535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113fa204-7b97-4bca-8609-9375a1a2e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.838216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fa204-7b97-4bca-8609-9375a1a2e89d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.838497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fa204-7b97-4bca-8609-9375a1a2e89d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.838809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113fa204-7b97-4bca-8609-9375a1a2e89d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.856727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:56.920437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502e843-1895-48e3-81c7-06a09dee6e22 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:56.941370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fa204-7b97-4bca-8609-9375a1a2e89d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:56.943761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113fa204-7b97-4bca-8609-9375a1a2e89d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:56.968207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab926d7c-3593-44bc-a20e-b51e7107034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:56.969646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab926d7c-3593-44bc-a20e-b51e7107034d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:56.970391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab926d7c-3593-44bc-a20e-b51e7107034d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:56.972463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab926d7c-3593-44bc-a20e-b51e7107034d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:56.993674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab926d7c-3593-44bc-a20e-b51e7107034d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ab926d7c-3593-44bc-a20e-b51e7107034d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:57.002092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:57.003307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:25:57.004266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:57.006651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 115/220] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:25:57.008230 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fc47e-daee-45fd-99c6-591042eea4b0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.008646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fc47e-daee-45fd-99c6-591042eea4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.009035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fc47e-daee-45fd-99c6-591042eea4b0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.010864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 116/221] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:57.015930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fc47e-daee-45fd-99c6-591042eea4b0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.099224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 106/222] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:25:57.100835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.101467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.101930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.105715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:57.125958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:57.127470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:57.150977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fc47e-daee-45fd-99c6-591042eea4b0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.176872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:57.192876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.194078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.194507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.195418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.196441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:57.196927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.199091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.212876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 107/224] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:57.213140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 117/224] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:57.214003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.229618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.336057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:57.336691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6853b-3ad1-456b-b874-792e0b05f7e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.340070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf980a96-e1e6-44e6-b025-813204f453b3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.375817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.376768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.376914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.377455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.377645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.377920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.378025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] PATH_INFO: `/v3/users/87aee27909464137b36ff8cae338ae46` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.378212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.379468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] RBAC: Authorizing `identity:get_user(user_id=87aee27909464137b36ff8cae338ae46)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.379580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.380970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.381101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.383100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 118/225] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/users/87aee27909464137b36ff8cae338ae46 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:57.383967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.391190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 108/226] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:57.392706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.438661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:57.494379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62d828-c04a-46e4-9a16-06b9cb265d22 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.501149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb74e65-de10-4e10-aab7-1c1bcd62eafc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.530928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.531865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.532227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.532559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.539193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.540259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.540604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.540940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.542253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.543736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.544239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:57.545833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 109/227] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:57.546381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:57.553124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:57.555075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:25:57.556979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 119/228] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:57.565332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.591938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:57.659162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d76bba-0e87-45ff-9cd3-3ecf13f7038f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.669491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:57.670327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b44cd-eeae-4d3a-a20e-ed711c3aabd2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.696673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.697253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.697531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.697801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.698731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=935ef8de15954dbeb2c80cd90e444b0e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:57.699068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.700702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.706798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 110/229] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/role_assignments?user.id=935ef8de15954dbeb2c80cd90e444b0e&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=3e228f026e734b8b9c06f578f2012163 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:57.711708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:57.712627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.713720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.714568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.715408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3/users/87aee27909464137b36ff8cae338ae46/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.720773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] RBAC: Authorizing `identity:create_grant(project_id=92768b6befe14406aed6530166b50ea3, user_id=87aee27909464137b36ff8cae338ae46, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.722640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.751640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 120/230] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:57 2022] PUT /identity/v3/projects/92768b6befe14406aed6530166b50ea3/users/87aee27909464137b36ff8cae338ae46/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:57.875954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f62af-9e72-4641-aa74-4213499d1c02 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:57.924671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:57.925618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:57.926032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:57.926433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:57.927782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:57.928220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:57.929799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:57.943659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 111/231] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:57.950401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:58.067471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:58.087973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42441292-528d-4fcc-b080-da2b9a48de0e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:58.127250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:58.128145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:58.128421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:58.128687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] PATH_INFO: `/v3/users/87aee27909464137b36ff8cae338ae46` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:58.129894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] RBAC: Authorizing `identity:get_user(user_id=87aee27909464137b36ff8cae338ae46)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:58.131399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:58.133332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 121/232] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:57 2022] GET /identity/v3/users/87aee27909464137b36ff8cae338ae46 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:58.139046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:58.189208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:58.280087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c04e39-2816-4d6e-af3c-adf853f79613 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:58.327612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56628698-27c9-428a-937d-c995d09c0ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:58.328488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56628698-27c9-428a-937d-c995d09c0ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:58.328758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56628698-27c9-428a-937d-c995d09c0ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:58.329020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56628698-27c9-428a-937d-c995d09c0ca1 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:58.330188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56628698-27c9-428a-937d-c995d09c0ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:58.331672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56628698-27c9-428a-937d-c995d09c0ca1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:58.333428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 112/233] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:25:58 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:25:58.339506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:58.482200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac692-4779-4dc8-bf27-4638339684b9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:58.539630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293209-3c1c-46a6-85f0-8a45b0907685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:58.540474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293209-3c1c-46a6-85f0-8a45b0907685 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:58.540753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293209-3c1c-46a6-85f0-8a45b0907685 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:58.541028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293209-3c1c-46a6-85f0-8a45b0907685 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3/users/87aee27909464137b36ff8cae338ae46/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:58.543382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87293209-3c1c-46a6-85f0-8a45b0907685 None admin] RBAC: Authorizing `identity:create_grant(project_id=92768b6befe14406aed6530166b50ea3, user_id=87aee27909464137b36ff8cae338ae46, role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:58.544934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87293209-3c1c-46a6-85f0-8a45b0907685 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:58.568211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 122/234] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:25:58 2022] PUT /identity/v3/projects/92768b6befe14406aed6530166b50ea3/users/87aee27909464137b36ff8cae338ae46/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:25:59.054183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e6b07-439b-44ef-90d9-35c15caad890 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.054701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e6b07-439b-44ef-90d9-35c15caad890 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.054701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e6b07-439b-44ef-90d9-35c15caad890 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:59.056177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 113/235] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:25:59.061925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.062396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.062467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:59.084183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:25:59.085455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:25:59.099761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:59.187151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 123/236] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:25:59.222287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e6b07-439b-44ef-90d9-35c15caad890 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:59.360016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e6b07-439b-44ef-90d9-35c15caad890 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:59.361828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:59.404064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:59.404942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.405373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.405652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:59.418146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:25:59.426190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:25:59.427437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:25:59.428196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 114/237] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:25:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:25:59.435081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:59.608652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b769c-9fe9-4407-a192-3522e45b3624 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:59.657362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:59.657956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.658232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.658501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:59.659351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:25:59.659653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:59.661112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:59.674665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 124/238] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:25:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:59.680871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:59.820933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833d01e-b0fa-417a-a895-dd1ecd9191b7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:25:59.863285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edea9d-f90a-42dc-a760-1a6abef0a7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:25:59.864217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edea9d-f90a-42dc-a760-1a6abef0a7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.864496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edea9d-f90a-42dc-a760-1a6abef0a7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.864765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edea9d-f90a-42dc-a760-1a6abef0a7f0 None admin] PATH_INFO: `/v3/users/803ecfbb4a8545bcb95491997a9eddaf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:59.886131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edea9d-f90a-42dc-a760-1a6abef0a7f0 None admin] RBAC: Authorizing `identity:get_user(user_id=803ecfbb4a8545bcb95491997a9eddaf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:25:59.887627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edea9d-f90a-42dc-a760-1a6abef0a7f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:25:59.889443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 115/239] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:25:59 2022] GET /identity/v3/users/803ecfbb4a8545bcb95491997a9eddaf => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:25:59.895268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:25:59.904981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:25:59.959333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c9a95-bd75-419b-af3e-7f54acb1b167 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.959502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c9a95-bd75-419b-af3e-7f54acb1b167 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.959782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c9a95-bd75-419b-af3e-7f54acb1b167 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:25:59.961161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 116/240] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:25:59.984660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:25:59.984964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:25:59.985194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.007872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:00.009429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:00.027516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:00.052652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d8c7f-7c0e-4773-b33c-9734086b317a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:00.108488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 117/241] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:00.127502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:00.128884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.129351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.129818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.138968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:00.271066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:00.272915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:00.285044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92e1b9-e799-44aa-bc86-d409187106ff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:00.289130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:00.335553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:00.336472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.336782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.337080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.338593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:00.340314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:00.342487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 118/242] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:00.348455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:00.370470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:00.430980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 125/243] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:25:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:00.432527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:00.461212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73520ef8-1a68-4b78-b727-22e297901c5b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:00.508136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d490fc6-3c5d-4e7a-bb3e-cad0d3da92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:00.509283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d490fc6-3c5d-4e7a-bb3e-cad0d3da92b7 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.509853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d490fc6-3c5d-4e7a-bb3e-cad0d3da92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.510348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d490fc6-3c5d-4e7a-bb3e-cad0d3da92b7 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.511400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d490fc6-3c5d-4e7a-bb3e-cad0d3da92b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:00.513435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d490fc6-3c5d-4e7a-bb3e-cad0d3da92b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:00.539354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 119/244] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:00 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:00.543773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e47-8c3e-4b54-b439-945e9e9a1263 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.544463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e47-8c3e-4b54-b439-945e9e9a1263 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.544937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27e47-8c3e-4b54-b439-945e9e9a1263 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.552498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 120/245] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:00.556833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.557382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.558139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.595756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:00.597470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:00.617172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8133e-fbf7-4356-8a0f-c52429017aa1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:00.632908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:00.699073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 121/246] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:00.726567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:00.730589 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:00.747609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430794-3652-4354-916b-7c824db7467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:00.748527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430794-3652-4354-916b-7c824db7467c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.748795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430794-3652-4354-916b-7c824db7467c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.749080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430794-3652-4354-916b-7c824db7467c None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.750421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430794-3652-4354-916b-7c824db7467c None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:00.751954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430794-3652-4354-916b-7c824db7467c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:00.753744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 126/247] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:00 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:00.759953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430794-3652-4354-916b-7c824db7467c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:00.868152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b35c36-bec0-40ca-81de-b0a42ebdb835 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:00.900993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430794-3652-4354-916b-7c824db7467c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:00.919006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e457710-ad6b-4cab-96f0-eb6b3c7ea6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:00.919877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457710-ad6b-4cab-96f0-eb6b3c7ea6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.920151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457710-ad6b-4cab-96f0-eb6b3c7ea6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.920412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e457710-ad6b-4cab-96f0-eb6b3c7ea6c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.933931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e457710-ad6b-4cab-96f0-eb6b3c7ea6c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9e457710-ad6b-4cab-96f0-eb6b3c7ea6c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:00.942621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:00.943528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:00.944283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 122/248] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:00.945412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f212-99e3-4bfa-820b-291e77f1e47f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.945505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f212-99e3-4bfa-820b-291e77f1e47f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.945759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f212-99e3-4bfa-820b-291e77f1e47f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.951237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 123/249] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:00.951237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706f212-99e3-4bfa-820b-291e77f1e47f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:00.952310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:00.953221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.953695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.954160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.955263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] RBAC: Adding query filter params (role.id=9b576d9657c34867877c2d8958bfa180, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=803ecfbb4a8545bcb95491997a9eddaf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:00.955776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:00.957452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314f3a8-611b-485a-ad95-27f8e584f7ef None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:00.965592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 127/250] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:00 2022] GET /identity/v3/role_assignments?user.id=803ecfbb4a8545bcb95491997a9eddaf&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=9b576d9657c34867877c2d8958bfa180 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:00.967282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:00.967560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:00.967843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:00.988054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:00.989277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:01.048373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.049728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.090358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 128/251] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:01.113795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706f212-99e3-4bfa-820b-291e77f1e47f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.128134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.160330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.161467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.161950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.162437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.163602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:01.164186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:01.166015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:01.177429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 124/252] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:01.184090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.248677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ccbe8-e79c-46c2-8eea-81a09c4f61be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.285526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.286628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.286783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.286996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.298496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:01.300008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593991d-ae9c-4cb1-b19f-b419fd947bcf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:01.307614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:01.308509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:01.309300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 129/253] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:01.315017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.331251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.340546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.341924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.342347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.342806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] PATH_INFO: `/v3/users/935ef8de15954dbeb2c80cd90e444b0e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.345134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] RBAC: Authorizing `identity:get_user(user_id=935ef8de15954dbeb2c80cd90e444b0e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:01.346844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:01.348347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.350131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 125/254] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/users/935ef8de15954dbeb2c80cd90e444b0e => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:01.357772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.423626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd686b0-1a86-4790-8744-af40ecba87b3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.458478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.459187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.459472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.459737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.460494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:01.460797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:01.462205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:01.471398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 130/255] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:01.476963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.538885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c42df-8f0b-47a2-b620-fe20dec4790f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.569239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.579245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3ab25-3e99-411d-9222-3c064dee95d1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.601281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af637475-936b-4133-ac26-d0d022fd568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.602174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af637475-936b-4133-ac26-d0d022fd568a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.602459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af637475-936b-4133-ac26-d0d022fd568a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.602814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af637475-936b-4133-ac26-d0d022fd568a None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.604463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af637475-936b-4133-ac26-d0d022fd568a None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:01.605959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af637475-936b-4133-ac26-d0d022fd568a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:01.612236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 126/256] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:01.613708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e1f1-8d51-47bc-af2b-d704bbd28fe6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.613992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e1f1-8d51-47bc-af2b-d704bbd28fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.614263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e1f1-8d51-47bc-af2b-d704bbd28fe6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.614658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89de31-a04e-48cb-802a-55a4a7e442f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.615577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89de31-a04e-48cb-802a-55a4a7e442f2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.615858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89de31-a04e-48cb-802a-55a4a7e442f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.615956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 127/257] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:01.616135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89de31-a04e-48cb-802a-55a4a7e442f2 None admin] PATH_INFO: `/v3/users/87aee27909464137b36ff8cae338ae46` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.617463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89de31-a04e-48cb-802a-55a4a7e442f2 None admin] RBAC: Authorizing `identity:get_user(user_id=87aee27909464137b36ff8cae338ae46)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:01.618004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806e1f1-8d51-47bc-af2b-d704bbd28fe6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.619070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89de31-a04e-48cb-802a-55a4a7e442f2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:01.620727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 131/258] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/users/87aee27909464137b36ff8cae338ae46 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:01.623663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.623949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.624225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.643046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:01.644245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:01.699093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.713532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 132/259] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:01.714405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.808572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806e1f1-8d51-47bc-af2b-d704bbd28fe6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.838240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:01.858404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5c38e-07b5-4232-b30f-6e052872c930 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:01.887973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.888877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.889155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.889424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3/users/935ef8de15954dbeb2c80cd90e444b0e/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:01.892608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=92768b6befe14406aed6530166b50ea3, user_id=935ef8de15954dbeb2c80cd90e444b0e, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:01.894072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:01.915430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:01.916103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:01.917316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 128/260] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:01 2022] PUT /identity/v3/projects/92768b6befe14406aed6530166b50ea3/users/935ef8de15954dbeb2c80cd90e444b0e/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:01.918146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:01.918512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:01.918816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:02.052692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:02.054192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:02.161069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08878e-f028-400e-8c7d-4705e4e68bb2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:02.191724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:02.208568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:02.219223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 133/261] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:02.226283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:02.242808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:02.248074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:02.248427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:02.248720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:02.275695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:02.293744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:02.295622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:02.297579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 129/262] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:02.303490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:02.437349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44ed20-e00d-43bc-80b7-1fa302fd72e8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:02.477613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:02.478959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:02.479245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:02.479519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:02.480294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:02.480599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:02.482067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:02.504458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16442a1-a8ce-4823-a2b0-4209c0d2b705 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:02.513151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 130/263] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:02.518742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:02.580107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:02.580990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:02.581267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:02.581535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:02.582902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:02.584400 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:02.586922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 134/264] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:02 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:02.595775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:02.605989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:02.701236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13fcb39-721d-4e0a-bf4b-5cc626163f18 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:02.728085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:02.764752 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:02.766039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:02.766524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:02.767006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] PATH_INFO: `/v3/users/87aee27909464137b36ff8cae338ae46` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:02.771935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] RBAC: Authorizing `identity:get_user(user_id=87aee27909464137b36ff8cae338ae46)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:02.774113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:02.776044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 131/265] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:02 2022] GET /identity/v3/users/87aee27909464137b36ff8cae338ae46 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:02.781621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:02.852907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c24a-f964-4783-a7b1-a414d2ac35a5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:02.940828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:02.941506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:02.941784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:02.942054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:02.942948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=87aee27909464137b36ff8cae338ae46) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:02.943263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:02.944736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:02.957472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 135/266] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:02 2022] GET /identity/v3/role_assignments?user.id=87aee27909464137b36ff8cae338ae46&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:02.983413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9b782-46ca-406b-909d-e20935181212 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:03.007532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:03.040867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:03.041533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:03.041856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:03.042129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:03.132886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:03.134365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:03.270034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 132/267] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:26:03.275346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:03.441712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f97d2-2a76-4c6f-97ec-b919776dad2a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:03.470464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:03.486791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cd959-0a7a-4531-b8ef-d2d2f23f438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:03.487705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd959-0a7a-4531-b8ef-d2d2f23f438e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:03.487990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd959-0a7a-4531-b8ef-d2d2f23f438e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:03.488251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd959-0a7a-4531-b8ef-d2d2f23f438e None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:03.489446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd959-0a7a-4531-b8ef-d2d2f23f438e None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:03.490950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd959-0a7a-4531-b8ef-d2d2f23f438e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:03.492980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 136/268] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:03 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:03.500419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:03.521574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:03.652676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6473c8-7d7a-4b3d-a3d7-a30123d14f54 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:03.653316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6473c8-7d7a-4b3d-a3d7-a30123d14f54 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:03.653852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6473c8-7d7a-4b3d-a3d7-a30123d14f54 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:03.655790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 137/269] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:03.659572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219aa11-2289-44a3-8c14-dd4a2df41535 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:03.661071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:03.661572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:03.662036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:03.693689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:03.695718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:03.723446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:03.724162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:03.724440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:03.724707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:03.725724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] RBAC: Adding query filter params (role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=87aee27909464137b36ff8cae338ae46) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:03.726038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:03.727539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:03.743092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 133/270] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:03 2022] GET /identity/v3/role_assignments?user.id=87aee27909464137b36ff8cae338ae46&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:03.811880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 138/271] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:03.845913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:03.944865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:03.979799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814bfd4-6fb3-4769-b633-2fc9c1581795 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.021851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.022667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.023043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.023345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.037033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:04.044920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:04.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:04.046520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 134/272] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:04.052422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.162975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed1398-64ad-450f-ac4d-2e4eb73c7afc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.182283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:04.204237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.204926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.205206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.205484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.206247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:04.206555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:04.207985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:04.217409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 139/273] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:04.222737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.329688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8120d4e-4833-40ae-8961-e2d6316cdb67 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.371208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abd5f5-c1dc-41d6-ae98-b9f2402e3bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.372134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abd5f5-c1dc-41d6-ae98-b9f2402e3bab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.372439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abd5f5-c1dc-41d6-ae98-b9f2402e3bab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.372757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abd5f5-c1dc-41d6-ae98-b9f2402e3bab None admin] PATH_INFO: `/v3/users/803ecfbb4a8545bcb95491997a9eddaf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.374390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abd5f5-c1dc-41d6-ae98-b9f2402e3bab None admin] RBAC: Authorizing `identity:get_user(user_id=803ecfbb4a8545bcb95491997a9eddaf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:04.375932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abd5f5-c1dc-41d6-ae98-b9f2402e3bab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:04.377768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 135/274] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:04 2022] GET /identity/v3/users/803ecfbb4a8545bcb95491997a9eddaf => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:04.383749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.490857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4928d5-c6c8-44f5-8336-1c2ac1a9f792 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.526222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.527139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.527423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.527703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.528969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:04.530456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:04.533679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e0f06-34c3-47f8-b4d5-b8bc83b8993c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.533960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e0f06-34c3-47f8-b4d5-b8bc83b8993c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.534316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e0f06-34c3-47f8-b4d5-b8bc83b8993c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.534714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 140/275] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:04 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:04.535854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 136/276] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:26:04.540568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.545675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14aa1a-12d6-4156-9c9d-a41a901578d3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.546073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14aa1a-12d6-4156-9c9d-a41a901578d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.546451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14aa1a-12d6-4156-9c9d-a41a901578d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.555348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:04.569750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ce14aa1a-12d6-4156-9c9d-a41a901578d3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:04.571233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce14aa1a-12d6-4156-9c9d-a41a901578d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:04.593983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:04.641405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 137/277] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:04.643215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3b0cf-3529-455f-8f5d-ad8b9e75f08c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.643682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3b0cf-3529-455f-8f5d-ad8b9e75f08c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.644163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3b0cf-3529-455f-8f5d-ad8b9e75f08c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.645609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 138/278] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:04.651236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.651523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.651800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.664983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151e0f-417e-44c8-8f5e-37a700eed89e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.668755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:04.670009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:04.706464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.707532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.707745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.707947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3/users/803ecfbb4a8545bcb95491997a9eddaf/roles/9b576d9657c34867877c2d8958bfa180` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.710166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] RBAC: Authorizing `identity:create_grant(project_id=92768b6befe14406aed6530166b50ea3, user_id=803ecfbb4a8545bcb95491997a9eddaf, role_id=9b576d9657c34867877c2d8958bfa180)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:04.711645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:04.726331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 141/279] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:04 2022] PUT /identity/v3/projects/92768b6befe14406aed6530166b50ea3/users/803ecfbb4a8545bcb95491997a9eddaf/roles/9b576d9657c34867877c2d8958bfa180 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:04.735086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.757373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 139/280] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:04.782555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.869565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:04.875661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde30a-f21e-45b3-8193-0e90719d0526 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.889121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:04.917625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c8f34-a5ba-40b1-b38a-699030754a7a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:04.929743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.930595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.930906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.931179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.949848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:04.957825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:04.958715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:04.959558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 142/281] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:04.966911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:04.967345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:04.968382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:04.969571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:04.970034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:04.971523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:04.973166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:04.975526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 140/282] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:04.981281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.093051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a016-7b1f-40dd-a63a-5d735178e5dc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.097217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ba185-7912-4d83-9cb8-02409a5e5d31 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.132839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.133428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.133704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.133978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.134732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:05.135080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:05.135992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:05.136497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219119b-17ea-4767-b402-f532860b72e3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:05.144851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39603a-2f7f-489b-8b8b-624d8e8b8b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.145533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39603a-2f7f-489b-8b8b-624d8e8b8b6a None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.145806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39603a-2f7f-489b-8b8b-624d8e8b8b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.146072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39603a-2f7f-489b-8b8b-624d8e8b8b6a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.146803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39603a-2f7f-489b-8b8b-624d8e8b8b6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:05.147578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 143/283] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:05.148251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39603a-2f7f-489b-8b8b-624d8e8b8b6a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:05.149290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0ccda-e204-4f47-82ed-be9a5d279a39 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.149763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0ccda-e204-4f47-82ed-be9a5d279a39 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.150036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0ccda-e204-4f47-82ed-be9a5d279a39 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.151481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 144/284] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:05.156241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0ccda-e204-4f47-82ed-be9a5d279a39 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.177187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 141/285] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:04 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:05.178289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.178577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.178869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.198407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:05.199675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:05.215554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:05.316627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 142/286] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:05.328753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0ccda-e204-4f47-82ed-be9a5d279a39 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.340962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.369797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.371085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.371595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.372050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] PATH_INFO: `/v3/users/8b4459a600bc4395a1cdecd1d457e067` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.389733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] RBAC: Authorizing `identity:get_user(user_id=8b4459a600bc4395a1cdecd1d457e067)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:05.391313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:05.393199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 145/287] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:05 2022] GET /identity/v3/users/8b4459a600bc4395a1cdecd1d457e067 => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:05.398948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.481364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b315e-2923-484c-94ff-e5e0127d7691 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.506166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:05.515219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:05.530108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684072dc-717f-4ee7-9b95-41300c05a61a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.530628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.531473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.533681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.533681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.548487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:05.559161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:05.560345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:05.561301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:05.564890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 143/288] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:05.571464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.581949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.582622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.582966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.583244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.584687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, user.id=8b4459a600bc4395a1cdecd1d457e067, scope.system=all) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:05.585003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:05.586861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:05.597930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 146/289] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 25 14:26:05 2022] GET /identity/v3/role_assignments?user.id=8b4459a600bc4395a1cdecd1d457e067&scope.system=all&role.id=3e228f026e734b8b9c06f578f2012163 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:05.697328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410c8cc-e472-45d6-9afa-d2068bcd524f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.735853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.736887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.737343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.737790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.738731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:05.739237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:05.740914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:05.752610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 144/290] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:05.758790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.876729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068dc570-548c-4b0e-8e7a-77e6deabacae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:05.912413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:05.913306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:05.913584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:05.913859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] PATH_INFO: `/v3/users/935ef8de15954dbeb2c80cd90e444b0e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:05.915093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] RBAC: Authorizing `identity:get_user(user_id=935ef8de15954dbeb2c80cd90e444b0e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:05.916548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:05.918195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 147/291] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:05 2022] GET /identity/v3/users/935ef8de15954dbeb2c80cd90e444b0e => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:05.923851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:05.973506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:06.042891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c11e8-72db-4175-a2c3-5233d4507d7d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:06.078624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151664e3-0667-463d-a17e-55846af13804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:06.079706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151664e3-0667-463d-a17e-55846af13804 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:06.080181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151664e3-0667-463d-a17e-55846af13804 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:06.080646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151664e3-0667-463d-a17e-55846af13804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:06.150124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151664e3-0667-463d-a17e-55846af13804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:06.152009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151664e3-0667-463d-a17e-55846af13804 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:06.197593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:06.257741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 145/292] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:26:06.263009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:06.302142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:06.404243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8b308-83cf-42ad-b591-964a6caf83ea None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:06.454326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efc004-c85e-4a0d-9620-6ac86515f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:06.455224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efc004-c85e-4a0d-9620-6ac86515f248 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:06.455500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efc004-c85e-4a0d-9620-6ac86515f248 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:06.455772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efc004-c85e-4a0d-9620-6ac86515f248 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:06.456984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efc004-c85e-4a0d-9620-6ac86515f248 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:06.458434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efc004-c85e-4a0d-9620-6ac86515f248 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:06.460411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 148/293] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:06 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:06.470233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151664e3-0667-463d-a17e-55846af13804 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:06.600368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151664e3-0667-463d-a17e-55846af13804 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:06.635077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:06.635871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:06.636141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:06.636411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:06.637262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=935ef8de15954dbeb2c80cd90e444b0e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:06.637567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:06.639197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc2d45-54e7-4435-b6a7-aaba5bb4a8fc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:06.645727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 146/294] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:06 2022] GET /identity/v3/role_assignments?user.id=935ef8de15954dbeb2c80cd90e444b0e&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=3e228f026e734b8b9c06f578f2012163 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:07.339576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0cac-b8ed-459a-b704-37712a434e05 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:07.339973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0cac-b8ed-459a-b704-37712a434e05 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:07.340170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0cac-b8ed-459a-b704-37712a434e05 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:07.341771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 149/295] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:07.347839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:07.348123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:07.348390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:07.354651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:07.368416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:07.369607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:07.449374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 147/296] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:07.472512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae0cac-b8ed-459a-b704-37712a434e05 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:07.564455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:07.577888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae0cac-b8ed-459a-b704-37712a434e05 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:07.613232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:07.614034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:07.614307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:07.614573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:07.625439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:07.633239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:07.634093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:07.634895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 150/297] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:07.640301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:07.743431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077b2e9-bf4f-4162-8023-f5c6bd86610b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:07.778909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:07.779932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:07.780400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:07.780841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:07.781796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:07.782270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:07.783855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:07.793241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 148/298] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:07.793928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:07.799656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:07.919796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e230b4c-1417-4fe1-8a7a-5caba6385a02 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:07.960318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1450d3-bba0-4bbb-9997-416882972557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:07.961293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1450d3-bba0-4bbb-9997-416882972557 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:07.961576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1450d3-bba0-4bbb-9997-416882972557 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:07.961859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1450d3-bba0-4bbb-9997-416882972557 None admin] PATH_INFO: `/v3/users/803ecfbb4a8545bcb95491997a9eddaf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:07.963151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1450d3-bba0-4bbb-9997-416882972557 None admin] RBAC: Authorizing `identity:get_user(user_id=803ecfbb4a8545bcb95491997a9eddaf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:07.964635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1450d3-bba0-4bbb-9997-416882972557 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:07.966766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 151/299] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:07 2022] GET /identity/v3/users/803ecfbb4a8545bcb95491997a9eddaf => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:07.972140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.048873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226757c9-f5dd-4a9c-8230-0ac4a21944d1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.049449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226757c9-f5dd-4a9c-8230-0ac4a21944d1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.050076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226757c9-f5dd-4a9c-8230-0ac4a21944d1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.053012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 152/300] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:08.058347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf1de-f673-4a28-9b92-af125f8eb795 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.058634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf1de-f673-4a28-9b92-af125f8eb795 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.058940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf1de-f673-4a28-9b92-af125f8eb795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.076748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-291cf1de-f673-4a28-9b92-af125f8eb795 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:08.077954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-291cf1de-f673-4a28-9b92-af125f8eb795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:08.090266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.095579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a640-81ee-42e9-add7-e5acc762ec8c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.131930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.132621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.132904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.133181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.151806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 153/301] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:08.160678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ddf6f-c8ca-47e6-bb21-663cf6541ceb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.161176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ddf6f-c8ca-47e6-bb21-663cf6541ceb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.161834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ddf6f-c8ca-47e6-bb21-663cf6541ceb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.163521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 154/302] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:08.169740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.170553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.171066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.189100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:08.190639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:08.210945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.219296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:08.220999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:08.265564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 155/303] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:08.271482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.338492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 149/304] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:08.343907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.352971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.379305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4378dd-f437-47de-adef-3a2ac925c587 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.417402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.418236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.418512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.418812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.429471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:08.437173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:08.438211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:08.439011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 156/305] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:08.439679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.447210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.453720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e9afa-03e8-4f35-a0e8-e48cc87bf811 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.486792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f38a7-4805-4647-8d44-473122ca1bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.487730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f38a7-4805-4647-8d44-473122ca1bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.488006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f38a7-4805-4647-8d44-473122ca1bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.488270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f38a7-4805-4647-8d44-473122ca1bd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.498179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f38a7-4805-4647-8d44-473122ca1bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fd7f38a7-4805-4647-8d44-473122ca1bd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:08.505778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:08.506633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 150/306] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:08.512464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986bc24-0cdc-4c27-95b2-dd14700631c2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.513179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986bc24-0cdc-4c27-95b2-dd14700631c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.513179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986bc24-0cdc-4c27-95b2-dd14700631c2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.513179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 151/307] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:08.513179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986bc24-0cdc-4c27-95b2-dd14700631c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.539015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16805c-5a74-4dc7-b3c9-efd7b0d89d2c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.574501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.575477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.575759 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.576032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.577244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:08.578717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:08.580458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 157/308] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:08.581220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.584656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.621783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986bc24-0cdc-4c27-95b2-dd14700631c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.654686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.655391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.655660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.655930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.656672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:08.656997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:08.658366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab714bc8-cce1-4cc1-9d56-dfea363104eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:08.666475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 152/309] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:08.667875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.668145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.668411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.680622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a80b2-802b-4a68-b473-6d4de89ca40b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.684701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:08.685883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:08.698615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.714393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.715099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.715350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.715652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.716435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:08.716733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:08.718110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:08.726056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 158/310] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:08.726887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.754189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 153/311] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:08.755323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.806448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:08.841538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c605ec2-8798-4853-97f6-7600c38ea157 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.856133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f479b-0af8-4566-aab9-387b03015dbf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:08.876058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.876720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.876997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.877267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.878111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] RBAC: Adding query filter params (role.id=9b576d9657c34867877c2d8958bfa180, scope.project.id=92768b6befe14406aed6530166b50ea3, user.id=803ecfbb4a8545bcb95491997a9eddaf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:08.878419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:08.882455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:08.888917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 159/312] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/role_assignments?user.id=803ecfbb4a8545bcb95491997a9eddaf&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=9b576d9657c34867877c2d8958bfa180 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:08.893840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:08.894736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:08.895008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:08.895311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] PATH_INFO: `/v3/users/d22d4b54de0444acaf5878addb1efa57` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:08.895733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.912244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] RBAC: Authorizing `identity:get_user(user_id=d22d4b54de0444acaf5878addb1efa57)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:08.913773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:08.915858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 154/313] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/users/d22d4b54de0444acaf5878addb1efa57 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:08.921543 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:08.932487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:09.013492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a0be9-9f33-4622-afb8-6e3630297b0a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:09.036709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30851866-d2f0-4924-87a7-b368f2095ebf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:09.045162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:09.053010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:09.053968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:09.054297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:09.054569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] PATH_INFO: `/v3/users/8b4459a600bc4395a1cdecd1d457e067` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:09.055846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] RBAC: Authorizing `identity:get_user(user_id=8b4459a600bc4395a1cdecd1d457e067)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:09.057339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:09.059097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 160/314] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/users/8b4459a600bc4395a1cdecd1d457e067 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:09.059964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:09.082372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:09.083776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:09.084207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:09.084474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:09.085665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:09.087132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:09.089022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 155/315] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:09.089552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:09.168413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:09.174346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8af18a-37b8-49bf-905d-4585a28f7e86 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:09.193868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3ebb2-b652-4641-b693-1ea8a36c0f8a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:09.206699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:09.207378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:09.207656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:09.207921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:09.208752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] RBAC: Adding query filter params (role.id=9b576d9657c34867877c2d8958bfa180, user.id=d22d4b54de0444acaf5878addb1efa57, scope.system=all) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:09.209056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:09.210503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:09.215360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 161/316] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 25 14:26:09 2022] GET /identity/v3/role_assignments?user.id=d22d4b54de0444acaf5878addb1efa57&scope.system=all&role.id=9b576d9657c34867877c2d8958bfa180 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:09.216251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:09.239223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a2293-2963-4bf4-9e0c-acc6768d8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:09.240526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2293-2963-4bf4-9e0c-acc6768d8f37 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:09.240966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2293-2963-4bf4-9e0c-acc6768d8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:09.241381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a2293-2963-4bf4-9e0c-acc6768d8f37 None admin] PATH_INFO: `/v3/system/users/8b4459a600bc4395a1cdecd1d457e067/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:09.243376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a2293-2963-4bf4-9e0c-acc6768d8f37 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8b4459a600bc4395a1cdecd1d457e067, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:09.245039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a2293-2963-4bf4-9e0c-acc6768d8f37 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:09.256963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 156/317] 10.222.0.32 () {58 vars in 1334 bytes} [Wed May 25 14:26:09 2022] PUT /identity/v3/system/users/8b4459a600bc4395a1cdecd1d457e067/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:09.310378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:09.360617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701e8de-c7ae-4f09-965f-38cd6de11275 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:09.400885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda349b0-7b7a-410b-afbf-c971c35a3646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:09.401528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda349b0-7b7a-410b-afbf-c971c35a3646 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:09.401809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda349b0-7b7a-410b-afbf-c971c35a3646 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:09.402082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda349b0-7b7a-410b-afbf-c971c35a3646 None admin] PATH_INFO: `/v3/groups` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:09.402832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda349b0-7b7a-410b-afbf-c971c35a3646 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:09.404422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda349b0-7b7a-410b-afbf-c971c35a3646 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:09.422971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 162/318] 10.222.0.32 () {60 vars in 1134 bytes} [Wed May 25 14:26:09 2022] POST /identity/v3/groups => generated 229 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:11.424393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d32305-0c8e-4d85-945b-212bd7c166c4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.424941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d32305-0c8e-4d85-945b-212bd7c166c4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.424941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d32305-0c8e-4d85-945b-212bd7c166c4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.426603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 157/319] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:11.432198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.432610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.432742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.450885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:11.452126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:11.485060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087999e-e804-4bdf-994b-501b40bed43e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.485333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087999e-e804-4bdf-994b-501b40bed43e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.485599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087999e-e804-4bdf-994b-501b40bed43e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.487157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 158/320] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:11.492941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.493216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.493486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.511920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:11.513163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:11.540167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 163/321] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:11.562441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:11.580613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 159/322] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:11.603513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:11.623001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:11.674814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:11.678522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5ef5-a0b1-470f-87fb-989be51cb731 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:11.709841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f3d2b-a46c-4024-a266-b1a7e698cb15 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:11.711895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032793f-3b0d-4e25-821a-c65587fbaef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:11.712737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032793f-3b0d-4e25-821a-c65587fbaef0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.713013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032793f-3b0d-4e25-821a-c65587fbaef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.713289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032793f-3b0d-4e25-821a-c65587fbaef0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.723675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032793f-3b0d-4e25-821a-c65587fbaef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c032793f-3b0d-4e25-821a-c65587fbaef0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:11.731560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:11.732442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:11.733313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 164/323] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:11.734103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d98e85-f123-46f1-9da0-d2d6d5224e66 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.734372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d98e85-f123-46f1-9da0-d2d6d5224e66 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.734647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d98e85-f123-46f1-9da0-d2d6d5224e66 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.735943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 165/324] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:11.738476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d98e85-f123-46f1-9da0-d2d6d5224e66 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:11.748869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099fe06-bf0b-4023-b75a-2c1b494b9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:11.749681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099fe06-bf0b-4023-b75a-2c1b494b9f2e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.749953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099fe06-bf0b-4023-b75a-2c1b494b9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.750218 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099fe06-bf0b-4023-b75a-2c1b494b9f2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.760125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099fe06-bf0b-4023-b75a-2c1b494b9f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4099fe06-bf0b-4023-b75a-2c1b494b9f2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:11.767732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:11.768621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:11.769376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 160/325] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:11.770033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.770347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.770567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.786837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:11.788078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:11.842980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d98e85-f123-46f1-9da0-d2d6d5224e66 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:11.854192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 161/326] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:11.855070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:11.856144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:11.879889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:11.880564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.880841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.881109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.881868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:11.882168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:11.883613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:11.895424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 166/327] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:11.895424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:11.913470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:11.955369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591e33-01e0-46fe-b95e-7df9c4952004 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:11.988744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:11.989420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:11.989692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:11.989982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:11.990727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:11.991060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:11.992450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:11.993301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883abf4-d44f-4058-9505-0d61b4a9ae84 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:12.000560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 162/328] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:12.001359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:12.029591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:12.030391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:12.030669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:12.030962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:12.032134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:12.033596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:12.035255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 167/329] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:12.036008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:12.078399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:12.104635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec15d56-e7c7-468a-b846-ee506597e2da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:12.124489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:12.138057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fcd74-2943-4829-90a8-26ccd9bbae4a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:12.138607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:12.139531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:12.139812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:12.140094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] PATH_INFO: `/v3/users/d22d4b54de0444acaf5878addb1efa57` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:12.141281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] RBAC: Authorizing `identity:get_user(user_id=d22d4b54de0444acaf5878addb1efa57)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:12.142717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:12.144384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 163/330] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:12 2022] GET /identity/v3/users/d22d4b54de0444acaf5878addb1efa57 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:12.145121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:12.177844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:12.178701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:12.179004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:12.179254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] PATH_INFO: `/v3/users/8b4459a600bc4395a1cdecd1d457e067` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:12.180453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] RBAC: Authorizing `identity:get_user(user_id=8b4459a600bc4395a1cdecd1d457e067)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:12.181912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:12.183732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 168/331] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:12 2022] GET /identity/v3/users/8b4459a600bc4395a1cdecd1d457e067 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:12.184370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:12.252673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3647c9-3587-4962-8480-239a8c71417b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:12.290919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:12.291206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e822d-16fa-4efa-ac67-8943a308d762 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:12.291591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:12.291866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:12.292135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:12.292828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:12.294274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:12.303066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:12.311683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 164/332] 10.222.0.32 () {60 vars in 1134 bytes} [Wed May 25 14:26:12 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:12.312508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:12.332543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e040c21-95a0-46bf-bfcf-a350b9adedd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:12.333493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040c21-95a0-46bf-bfcf-a350b9adedd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:12.333775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040c21-95a0-46bf-bfcf-a350b9adedd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:12.334045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040c21-95a0-46bf-bfcf-a350b9adedd5 None admin] PATH_INFO: `/v3/system/users/d22d4b54de0444acaf5878addb1efa57/roles/9b576d9657c34867877c2d8958bfa180` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:12.335794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e040c21-95a0-46bf-bfcf-a350b9adedd5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d22d4b54de0444acaf5878addb1efa57, role_id=9b576d9657c34867877c2d8958bfa180)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:12.337225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e040c21-95a0-46bf-bfcf-a350b9adedd5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:12.348569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 169/333] 10.222.0.32 () {58 vars in 1334 bytes} [Wed May 25 14:26:12 2022] PUT /identity/v3/system/users/d22d4b54de0444acaf5878addb1efa57/roles/9b576d9657c34867877c2d8958bfa180 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:12.369999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:12.442671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22a04a-ba3c-4a51-905a-1418fcbec840 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:12.488176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:12.489206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:12.489699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:12.490146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:12.491280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] RBAC: Adding query filter params (role.id=3e228f026e734b8b9c06f578f2012163, user.id=8b4459a600bc4395a1cdecd1d457e067, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:12.491797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:12.493549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dea73-07d1-4444-9a10-5383b84d7129 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:12.508327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 165/334] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 25 14:26:12 2022] GET /identity/v3/role_assignments?user.id=8b4459a600bc4395a1cdecd1d457e067&scope.system=all&role.id=3e228f026e734b8b9c06f578f2012163 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:14.729098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190921-abc1-46df-b637-c822c10cc206 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:14.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190921-abc1-46df-b637-c822c10cc206 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:14.731687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190921-abc1-46df-b637-c822c10cc206 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:14.734662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 170/335] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:14.741722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdfd2a-a008-4f4e-adf7-7c247d64b7b6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:14.743368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdfd2a-a008-4f4e-adf7-7c247d64b7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:14.744162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdfd2a-a008-4f4e-adf7-7c247d64b7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:14.783121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4cdfd2a-a008-4f4e-adf7-7c247d64b7b6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:14.785666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4cdfd2a-a008-4f4e-adf7-7c247d64b7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:14.904518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 166/336] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:14.951963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190921-abc1-46df-b637-c822c10cc206 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.035578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318ec2d-c306-44c8-a4d2-3f1c354abeda None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.036201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318ec2d-c306-44c8-a4d2-3f1c354abeda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.036688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318ec2d-c306-44c8-a4d2-3f1c354abeda None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.039777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 167/337] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:15.044087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d3d0d-0db3-461d-ba31-bfa76913ac7f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.044633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d3d0d-0db3-461d-ba31-bfa76913ac7f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.045109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d3d0d-0db3-461d-ba31-bfa76913ac7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.077492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-703d3d0d-0db3-461d-ba31-bfa76913ac7f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:15.088594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-703d3d0d-0db3-461d-ba31-bfa76913ac7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:15.120643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190921-abc1-46df-b637-c822c10cc206 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.141293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:15.214972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4df9ff-a211-4e7a-bb32-a70be2de0c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:15.215862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4df9ff-a211-4e7a-bb32-a70be2de0c2f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.216142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4df9ff-a211-4e7a-bb32-a70be2de0c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.216421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4df9ff-a211-4e7a-bb32-a70be2de0c2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.236733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4df9ff-a211-4e7a-bb32-a70be2de0c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6e4df9ff-a211-4e7a-bb32-a70be2de0c2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:15.244451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:15.245529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 14:26:15.246431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:15.249071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 171/338] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:15.250584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a6fef-6f6e-4312-b12b-0d2de6829007 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.251066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a6fef-6f6e-4312-b12b-0d2de6829007 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.251531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a6fef-6f6e-4312-b12b-0d2de6829007 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.252979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 172/339] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:15.256032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 168/340] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:15.258257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a6fef-6f6e-4312-b12b-0d2de6829007 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.264228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.264731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.265220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.286687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:15.288146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:15.318187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:15.429457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 169/341] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:15.430601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.432295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a6fef-6f6e-4312-b12b-0d2de6829007 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.482370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:15.483087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.483373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.483646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.484409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:15.484715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:15.491657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:15.513448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 173/342] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:15.514382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.576009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:15.583138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dc774-61b3-4b31-b0da-5050f56ac6f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.624541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:15.625498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.625826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.626103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.640224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:15.649107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:15.650235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:15.651237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:15.656578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 170/343] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:15.657778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.722448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:15.741779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997c7c8-220e-4f1c-be0a-41315442cf77 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.782559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f788e7c-c304-40a2-b9a8-d8aca4bb7522 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.785235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:15.786376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.786885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.787347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.800492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:15.808414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:15.809555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:15.810345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:15.814852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 174/344] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:15.815873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.822157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:15.823193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.823479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.823761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] PATH_INFO: `/v3/users/d22d4b54de0444acaf5878addb1efa57` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.825087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] RBAC: Authorizing `identity:get_user(user_id=d22d4b54de0444acaf5878addb1efa57)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:15.826614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:15.829806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 171/345] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:26:15 2022] GET /identity/v3/users/d22d4b54de0444acaf5878addb1efa57 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:15.830970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:15.860938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:15.947334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cc65e-0022-4093-a13e-28d342a425ab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.961630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af001f9-82d2-4e41-bed1-6aa12cf15a9d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:15.986318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:15.987501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:15.988054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:15.988501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:15.989463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:15.989952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:15.991663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.002112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 172/346] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:16.003659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.005412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.006093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.006375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.006679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.007483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:16.007793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:16.009196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd475f-988f-43c8-bfab-b3bee832d916 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.018810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 175/347] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:16.020377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72f4a-2ced-4c55-ad97-82bf3f245f94 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.020845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72f4a-2ced-4c55-ad97-82bf3f245f94 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.021314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb72f4a-2ced-4c55-ad97-82bf3f245f94 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.022929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 176/348] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:16.023850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb72f4a-2ced-4c55-ad97-82bf3f245f94 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.044662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.110530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.128247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8107bfe9-e294-42dc-8ef1-76766dd158b0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:16.134912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb72f4a-2ced-4c55-ad97-82bf3f245f94 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:16.168095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.169062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.169593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.170087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.171300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] RBAC: Adding query filter params (role.id=9b576d9657c34867877c2d8958bfa180, user.id=d22d4b54de0444acaf5878addb1efa57, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:16.171984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:16.173849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.174119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff385e-4b9b-4672-a86a-4d6b55e99d28 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.174508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.174835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.175112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.189924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 173/349] 10.222.0.32 () {56 vars in 1308 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/role_assignments?user.id=d22d4b54de0444acaf5878addb1efa57&scope.system=all&role.id=9b576d9657c34867877c2d8958bfa180 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:16.191841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.192374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.192925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.229381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:16.232749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:16.320187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:16.322092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.347294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.407834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 174/350] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:16.412388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.496630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 177/351] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:16.497546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.507785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.536010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea340a-2344-41b9-bb4c-fc9ea652c96e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:16.579324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.580359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.580854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.581316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.616645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc529b3c-f6e9-41dc-bf26-040200581ae9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:16.655393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.656313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.656598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.656874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.657932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.671463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:16.674281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:16.676370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:16.681705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:16.682613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:16.683898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 178/352] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:16.684752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.762678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.790769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 175/353] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:16.792689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.798352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73d1b9-c668-4c93-9d7a-9d4f7d76d195 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:16.835601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.836665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.837030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.837401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.838857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:16.840633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.842601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 179/354] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:16.843507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.904647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:16.911751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101c533-819d-4b98-97eb-ec3c288613b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:16.949142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:16.950272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:16.950951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:16.951464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:16.952547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:16.953054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:16.954631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:16.964315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 176/355] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:16.965641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:16.967091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435b7aa-ab8d-4f18-a564-211351d54cd0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.004983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.007710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.009011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.009464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.009888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.011285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.013888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.016319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 180/356] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:17.017272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.080260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d48801-dd74-4b25-adbf-162c0f2398aa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.120246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.121295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.121651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.121994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.130308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d77073-7fe7-474b-9911-097441e630ea None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.131620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.134493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.136339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 177/357] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:16 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:17.137125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.151091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.183922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.184960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.185455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.185970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.273095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.277168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdd931-8a13-4458-8b05-9352de19f2b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.280052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.280929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.314741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ba92-0bdb-47d1-a5f3-a9db4d2584cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.315639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ba92-0bdb-47d1-a5f3-a9db4d2584cd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.315919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ba92-0bdb-47d1-a5f3-a9db4d2584cd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.316381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ba92-0bdb-47d1-a5f3-a9db4d2584cd None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.317348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ba92-0bdb-47d1-a5f3-a9db4d2584cd None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.319425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ba92-0bdb-47d1-a5f3-a9db4d2584cd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.320427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 178/358] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:17.321613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02019fc-c92c-490a-9a8f-fe01f1b8064d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.321890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02019fc-c92c-490a-9a8f-fe01f1b8064d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.322177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02019fc-c92c-490a-9a8f-fe01f1b8064d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.327533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 179/359] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:17.327533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02019fc-c92c-490a-9a8f-fe01f1b8064d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.433366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 181/360] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:17.434965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.451698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.461542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02019fc-c92c-490a-9a8f-fe01f1b8064d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.504279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.504946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.505232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.505505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.506350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=141ee3a6fe3b41d5a050afe46844c163) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:17.506661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.508201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e04e1c-4a87-4f1a-87e9-a0dcae9722f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.515434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 180/361] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=3e228f026e734b8b9c06f578f2012163 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:17.517899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9774a8-04f2-4bf3-a9e0-921eb6e18f71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.518183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9774a8-04f2-4bf3-a9e0-921eb6e18f71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.518456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9774a8-04f2-4bf3-a9e0-921eb6e18f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.539015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a9774a8-04f2-4bf3-a9e0-921eb6e18f71 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:17.540509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a9774a8-04f2-4bf3-a9e0-921eb6e18f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:17.559863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060476-a6a4-4576-af49-f56ec87b95d8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.580665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.606602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.607354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.607637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.607910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.609132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=141ee3a6fe3b41d5a050afe46844c163) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:17.609324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.611430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99ff3d-d30e-417f-bb4d-7510fd6730cb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.622726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 182/362] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:17.624760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0081-27c4-4da8-a3b2-875acd45c960 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.625307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0081-27c4-4da8-a3b2-875acd45c960 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.625804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0081-27c4-4da8-a3b2-875acd45c960 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.627978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 183/363] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:17.629078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f0081-27c4-4da8-a3b2-875acd45c960 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.652821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 181/364] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:17.653907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.654245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.654523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.679307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:17.680811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:17.743189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.775072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f0081-27c4-4da8-a3b2-875acd45c960 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.790130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 182/365] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:17.793889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.834945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.836565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.837097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.837596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.840060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:17.842096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:17.844667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 184/366] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:17.845799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:17.926982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:17.934322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acfad4-437c-4127-a78f-7390541a6c8a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.972327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32247b4-799d-4d48-9df0-21c469718126 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:17.978806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:17.979640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:17.979923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:17.980194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:17.994226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:18.004284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:18.005206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:18.006007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 183/367] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:18.006656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.020510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.021720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.022197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.022711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.040232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:18.052120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:18.053044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:18.053816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 185/368] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:18.054481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.066508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:18.138789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84308c73-2f09-49a7-8ac4-f0fca0881fa1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.180152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.180406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3f968-ce88-453f-bb53-25166841a210 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.181599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.182850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.183387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.184872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:18.186548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:18.188875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 184/369] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:18.189963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.219232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:18.225598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.226559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.226869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.227146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.228546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:18.228867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:18.230562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:18.245574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 186/370] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:18.246360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.329464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93fccf-8447-48ce-b212-90a39abf33fc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.338556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:18.372953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.373638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.373916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.374185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.374976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:18.375278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:18.376656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:18.382080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b099010-d6c4-4df9-9919-ae01ef93d6ba None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.388020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 185/371] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:18.388880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.430252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.430979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.431251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.431534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.432398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=92768b6befe14406aed6530166b50ea3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:18.432718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:18.434848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:18.443070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 187/372] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=3e228f026e734b8b9c06f578f2012163 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:18.444169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.540371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:18.565124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f07b9-ca81-4395-b9b2-26408ab7314d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.599267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:18.606183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.606788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.607276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.607380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.611982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab027d-d0e5-423c-af87-72791738b956 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.689447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:18.690978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:18.724590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.725312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.725587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.725851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.808759 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 188/373] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:18.815645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:18.842537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:18.844159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:18.851066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:18.930147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e800b-03d9-4bec-ba85-b96bc596d9e6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:18.995478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:18.996379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:18.996690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:18.996997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] PATH_INFO: `/v3/projects/9157b1edcbb3430198b55709b5ddfe19` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:18.998977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:19.000075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] RBAC: Authorizing `identity:get_project(project_id=9157b1edcbb3430198b55709b5ddfe19)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:19.001555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:19.003378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 189/374] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/projects/9157b1edcbb3430198b55709b5ddfe19 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:19.010914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:19.038845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 186/375] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:19.042149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:19.199689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:19.204590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31fedc-825f-45da-8028-4839621604d4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:19.217569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad84e37-737d-41ed-842f-afad3edd3d74 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:19.274617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:19.275581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:19.275861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:19.276136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] PATH_INFO: `/v3/groups/dac9783cb06040668d7f818ca521c7d8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:19.278818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:19.279689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:19.279966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:19.280235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:19.282906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:19.284378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:19.286106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 187/376] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:19 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:19.291403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] RBAC: Authorizing `identity:get_group(group_id=dac9783cb06040668d7f818ca521c7d8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:19.292886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:19.294857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 190/377] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:19 2022] GET /identity/v3/groups/dac9783cb06040668d7f818ca521c7d8 => generated 229 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:19.295635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:19.301379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:19.427125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:19.434033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630d1e9-bd43-4a4a-886e-6f881fe33406 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:19.472037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:19.472622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:19.472904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:19.473181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:19.474036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] RBAC: Adding query filter params (group.id=dac9783cb06040668d7f818ca521c7d8, role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=9157b1edcbb3430198b55709b5ddfe19) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:19.474351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:19.478214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:19.485076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 191/378] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:19 2022] GET /identity/v3/role_assignments?group.id=dac9783cb06040668d7f818ca521c7d8&scope.project.id=9157b1edcbb3430198b55709b5ddfe19&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:19.513224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b8c08-4302-4a24-8391-4ecc07ba4a06 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:19.550312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f10710-3daf-478d-92fd-97f71a9bcdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:19.551553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f10710-3daf-478d-92fd-97f71a9bcdab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:19.551890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f10710-3daf-478d-92fd-97f71a9bcdab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:19.552162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f10710-3daf-478d-92fd-97f71a9bcdab None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:19.553527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f10710-3daf-478d-92fd-97f71a9bcdab None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:19.555041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f10710-3daf-478d-92fd-97f71a9bcdab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:19.557100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 188/379] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:19 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:19.563183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:19.604381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:19.700542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4c052-b1a4-4703-ae30-3d8d095e6a80 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:19.738691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:19.739369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:19.739654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:19.739928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:19.740846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=92768b6befe14406aed6530166b50ea3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:19.741162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:19.742670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95956108-fbd0-4679-99ad-6ba5eb5bcb94 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:19.749711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 192/380] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:19 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:20.219871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d1de-3283-4835-a976-1ac5f331effd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.220411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d1de-3283-4835-a976-1ac5f331effd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.220683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4d1de-3283-4835-a976-1ac5f331effd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.222315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 189/381] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:20.228192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95797d-81a6-4e7f-9c5a-44b8bf915052 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.229214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95797d-81a6-4e7f-9c5a-44b8bf915052 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.229696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95797d-81a6-4e7f-9c5a-44b8bf915052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.248695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3c95797d-81a6-4e7f-9c5a-44b8bf915052 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:20.250257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3c95797d-81a6-4e7f-9c5a-44b8bf915052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:20.328887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 193/382] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:20.351430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff439c3-7942-4daf-8453-56e3a92b75c3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.352047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff439c3-7942-4daf-8453-56e3a92b75c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.352509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff439c3-7942-4daf-8453-56e3a92b75c3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.354104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 190/383] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:20.359667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff439c3-7942-4daf-8453-56e3a92b75c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:20.359841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.360065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.360248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.380139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:20.381347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:20.410906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:20.509656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 194/384] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:20.518463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff439c3-7942-4daf-8453-56e3a92b75c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:20.550398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:20.552110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:20.555816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:20.558008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.558295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.558623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.570945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:20.581205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:20.582323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:20.583180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:20.584314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 191/385] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:20.590622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:20.700172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2fabf-2a64-4beb-aab3-a3cbe3a11de8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:20.739413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:20.740825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.741328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.741816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.754539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:20.763156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:20.764062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:20.764849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:20.767790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 195/386] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:20.774560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:20.777131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f1769-fbc1-416f-9f24-18f42356b511 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:20.806779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:20.832116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:20.832823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.833106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.833381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.834151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:20.834459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:20.835903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:20.850745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 192/387] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:20.863865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:20.906610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a00f68-fb09-48ae-b4b8-8389491a1635 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:20.927262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:20.948946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:20.949635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:20.949989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:20.950265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:20.951059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:20.951364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:20.952837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:20.962412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 196/388] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:20.968126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:20.997496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d8b7b-3b44-4309-8e48-5e7d19bd3c0b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.044466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.045267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.045546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.045819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.047081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:21.049142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:21.051549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 193/389] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:20 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:21.058460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:21.127633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:21.190160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499c83c-1994-4475-b5a0-59960cccbd26 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.233404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.234290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.234800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.235376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.237041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:21.238702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:21.244304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 197/390] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:20 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:21.251308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:21.270638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79966e72-bc1e-4c4d-b51a-d91f6097dc4d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.288669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:21.331144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.332504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.333103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.333596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.335500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:21.337917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:21.341362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 194/391] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:21 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:21.347732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:21.448058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0d147-7262-4218-a49f-7f60485d516a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.494364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e80c4-d501-428c-88fb-302e7dc432df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.495362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e80c4-d501-428c-88fb-302e7dc432df None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.495643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e80c4-d501-428c-88fb-302e7dc432df None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.495914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e80c4-d501-428c-88fb-302e7dc432df None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.497277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e80c4-d501-428c-88fb-302e7dc432df None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:21.498897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e80c4-d501-428c-88fb-302e7dc432df None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:21.500986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 198/392] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:21 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:21.502340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250382f9-aa70-40bd-9493-93ef1cb9cca7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.502624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250382f9-aa70-40bd-9493-93ef1cb9cca7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.504876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250382f9-aa70-40bd-9493-93ef1cb9cca7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.506796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 199/393] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:21.512074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.512593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.513109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.518538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc1f75-a82c-461f-b396-03188c0c16e5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.534691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:21.536356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:21.540566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:21.563744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.564908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.565189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.565465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163/groups/d0f965611ab245fd865708205d2b5d1f/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.568355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=141ee3a6fe3b41d5a050afe46844c163, group_id=d0f965611ab245fd865708205d2b5d1f, role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:21.570172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:21.594882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 195/394] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 25 14:26:21 2022] PUT /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163/groups/d0f965611ab245fd865708205d2b5d1f/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:26:21.594882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:21.632479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:21.649463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 200/395] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:21.697523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:21.720204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd7771-6e8c-48c0-8d95-430724ce4cc4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.762615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.763557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.763893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.764174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163/groups/d0f965611ab245fd865708205d2b5d1f/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.771908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] RBAC: Authorizing `identity:create_grant(project_id=141ee3a6fe3b41d5a050afe46844c163, group_id=d0f965611ab245fd865708205d2b5d1f, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:21.773498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:21.802745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 196/396] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 25 14:26:21 2022] PUT /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163/groups/d0f965611ab245fd865708205d2b5d1f/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:21.887950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fee624-3a5a-4f4b-bec2-2429fb6048a4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:21.933145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:21.933985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:21.934303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:21.934575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:21.944475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:21.948734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:21.957711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:21.958641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:21.959459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 201/397] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:21.966818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:22.108131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:22.113888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c38b49-9ad6-4324-95d0-d59b2283a59f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:22.153809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:22.154639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.155044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.155332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.156110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:22.156419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:22.157811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:22.170934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 197/398] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:22.174636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:22.387162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137006a0-fec3-4ec6-a15e-a50ace0a1e68 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:22.438208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b8e22-c221-483e-8eab-538fceb85434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:22.439093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8e22-c221-483e-8eab-538fceb85434 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.439373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8e22-c221-483e-8eab-538fceb85434 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.439641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b8e22-c221-483e-8eab-538fceb85434 None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.444160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8e22-c221-483e-8eab-538fceb85434 None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:22.445722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b8e22-c221-483e-8eab-538fceb85434 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:22.447920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 202/399] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:22 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:22.453902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:22.595621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1949d1-0c8c-4a7f-89e1-66635647546a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:22.637745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:22.638647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.638970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.639238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.640364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e53d5-9ba8-4bd1-a8a8-7acf895b9a37 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.640495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:22.640665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e53d5-9ba8-4bd1-a8a8-7acf895b9a37 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.640923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e53d5-9ba8-4bd1-a8a8-7acf895b9a37 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.641888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:22.642646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 203/400] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:22.643856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 198/401] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:22 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:22.651222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc7149-de4c-45f7-a522-2ccd5eeb13e8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.651497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc7149-de4c-45f7-a522-2ccd5eeb13e8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.651776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc7149-de4c-45f7-a522-2ccd5eeb13e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.652128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:22.678262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1cbc7149-de4c-45f7-a522-2ccd5eeb13e8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:22.679834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1cbc7149-de4c-45f7-a522-2ccd5eeb13e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:22.694249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:22.760480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:22.767959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 204/402] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:22.769456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafd074-7e4b-49e9-9e29-3d6f886c7c33 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.769803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafd074-7e4b-49e9-9e29-3d6f886c7c33 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.770702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafd074-7e4b-49e9-9e29-3d6f886c7c33 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.772618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 205/403] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:22.778084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.778369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.778637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.785251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a417-0395-4ea3-8dea-3a0a3c3a6a8c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:22.806908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:22.808423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:22.822313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:22.823326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:22.823676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:22.824067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3/groups/d0f965611ab245fd865708205d2b5d1f/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:22.826399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] RBAC: Authorizing `identity:create_grant(project_id=92768b6befe14406aed6530166b50ea3, group_id=d0f965611ab245fd865708205d2b5d1f, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:22.827886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:22.849825 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 199/404] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 25 14:26:22 2022] PUT /identity/v3/projects/92768b6befe14406aed6530166b50ea3/groups/d0f965611ab245fd865708205d2b5d1f/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:22.850875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:22.928340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 206/405] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:22.979093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.020348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:23.054082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61301480-49a0-46f8-94f2-386b1fbb2a32 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:23.121054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:23.122824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:23.123327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:23.123816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:23.147068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd009ad-3340-4904-a902-0287de3ca7f3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:23.151623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:23.164747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:23.166565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:23.168199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 200/406] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:23.172733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.190517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:23.200686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:23.201494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:23.201773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:23.202037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:23.216634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:23.232122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:23.233848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:23.240094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 207/407] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:23.259679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.400069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be1a46-4613-4e8a-aee4-c43d8438434c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:23.444455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:23.445578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:23.446059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:23.446529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:23.447687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:23.448196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:23.450223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:23.460694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 201/408] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:23.467342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.491228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:23.499921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245e0f2-631f-4886-96c8-e0517c96d8ce None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:23.573577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:23.574684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:23.575477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:23.575955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:23.577081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:23.577615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:23.579269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:23.590125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 208/409] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:23.597050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.671398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:23.710841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ecff-7439-4e5a-917a-f2492d86cada None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:23.750078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:23.750898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:23.751183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:23.751456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:23.752646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:23.754105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:23.756079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 202/410] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:23 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:23.761083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.787739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fa6a1-08c5-494e-8760-0286297b40ae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:23.850044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:23.850950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:23.851236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:23.851509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] PATH_INFO: `/v3/groups/dac9783cb06040668d7f818ca521c7d8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:23.856399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] RBAC: Authorizing `identity:get_group(group_id=dac9783cb06040668d7f818ca521c7d8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:23.861917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:23.864640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 209/411] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:23 2022] GET /identity/v3/groups/dac9783cb06040668d7f818ca521c7d8 => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:23.872129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:23.917988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:23.973991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b0c60-50e5-4772-bd69-2265111034b5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:24.012401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:24.013202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.013480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.013751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.016210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:24.017663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:24.021322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 203/412] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:23 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:24.026197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:24.084471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370f3b7-f137-4703-a650-dc2b19f68f38 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:24.119148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:24.157736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:24.158624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.158937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.159209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] PATH_INFO: `/v3/projects/9157b1edcbb3430198b55709b5ddfe19` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.161391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] RBAC: Authorizing `identity:get_project(project_id=9157b1edcbb3430198b55709b5ddfe19)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:24.162884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:24.167270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 210/413] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:23 2022] GET /identity/v3/projects/9157b1edcbb3430198b55709b5ddfe19 => generated 344 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:24.173650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:24.226354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8eea7-0bd9-42c9-8578-90087e196422 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:24.311641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ee3ec-9654-46b3-a43d-a25e4cb13e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:24.312480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee3ec-9654-46b3-a43d-a25e4cb13e0d None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.312759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee3ec-9654-46b3-a43d-a25e4cb13e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.313032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ee3ec-9654-46b3-a43d-a25e4cb13e0d None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3/groups/d0f965611ab245fd865708205d2b5d1f/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.319555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ee3ec-9654-46b3-a43d-a25e4cb13e0d None admin] RBAC: Authorizing `identity:create_grant(project_id=92768b6befe14406aed6530166b50ea3, group_id=d0f965611ab245fd865708205d2b5d1f, role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:24.321012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ee3ec-9654-46b3-a43d-a25e4cb13e0d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:24.326497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffbee4-ea45-490e-9374-a8c179345832 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:24.342528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 204/414] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 25 14:26:24 2022] PUT /identity/v3/projects/92768b6befe14406aed6530166b50ea3/groups/d0f965611ab245fd865708205d2b5d1f/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:26:24.344008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a664-6e18-4e54-b85d-50839a89bdfe None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.344302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a664-6e18-4e54-b85d-50839a89bdfe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.344579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04a664-6e18-4e54-b85d-50839a89bdfe None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.346992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 205/415] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:24.352398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.353368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.353817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.374095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:24.386649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:24.388957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.389987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:24.390912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.391241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] PATH_INFO: `/v3/projects/9157b1edcbb3430198b55709b5ddfe19/groups/dac9783cb06040668d7f818ca521c7d8/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.393280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:24.394930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] RBAC: Authorizing `identity:create_grant(project_id=9157b1edcbb3430198b55709b5ddfe19, group_id=dac9783cb06040668d7f818ca521c7d8, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:24.398187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:24.421787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 211/416] 10.222.0.32 () {58 vars in 1442 bytes} [Wed May 25 14:26:24 2022] PUT /identity/v3/projects/9157b1edcbb3430198b55709b5ddfe19/groups/dac9783cb06040668d7f818ca521c7d8/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:24.495042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 206/417] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:24.533821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:24.602651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:24.667504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20a2f-43de-4e66-917a-e13194664f8b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:24.706506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c344d1e-3550-4af8-8a48-42f9caf1a330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:24.707379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c344d1e-3550-4af8-8a48-42f9caf1a330 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.707653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c344d1e-3550-4af8-8a48-42f9caf1a330 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.707918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c344d1e-3550-4af8-8a48-42f9caf1a330 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.720060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c344d1e-3550-4af8-8a48-42f9caf1a330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2c344d1e-3550-4af8-8a48-42f9caf1a330 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:24.728426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:24.729346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:24.730096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 212/418] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:24.736063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:24.742483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f5e5-c960-4213-b430-d8f006f205a8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.742803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f5e5-c960-4213-b430-d8f006f205a8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.743065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f5e5-c960-4213-b430-d8f006f205a8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.744647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 213/419] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:26:24.752264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.752543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.752809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.770389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:24.771791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:24.877603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 214/420] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:24.908738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa775-0774-49c9-962f-563c1d3324fa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:24.927011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:24.985472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:24.986170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:24.986450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:24.986706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:24.986826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:24.987518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:24.987824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:24.989333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:24.991019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:25.006296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 207/421] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:25.011794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:25.093482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b94ec4-0af7-4881-89d3-2f6ae75fe633 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:25.132042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:25.132776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.133061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.133331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.144536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:25.152405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:25.153362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:25.154149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 215/422] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:25.160847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:25.198515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496665d6-df22-4f00-85d7-acab5c0581b2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:25.294034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fb0f0-c1a0-4bd2-8032-ea9c278e1ac6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:25.299471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661bd20e-2d2e-491f-bba5-78c5b49b59e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:25.300392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bd20e-2d2e-491f-bba5-78c5b49b59e4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.301208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bd20e-2d2e-491f-bba5-78c5b49b59e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.301696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661bd20e-2d2e-491f-bba5-78c5b49b59e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.328122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:25.331784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:25.332485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.332764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.334707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.336982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:25.337309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:25.338767 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:25.354922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 216/423] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:25.369586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:25.405366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:25.418842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bd20e-2d2e-491f-bba5-78c5b49b59e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:25.420401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661bd20e-2d2e-491f-bba5-78c5b49b59e4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:25.534066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2752d-f5c1-4935-83cc-2737f84c3ef7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:25.577270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1de7c5-4f64-4574-864c-e2ff36bcda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:25.577941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1de7c5-4f64-4574-864c-e2ff36bcda86 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.578223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1de7c5-4f64-4574-864c-e2ff36bcda86 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.578498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1de7c5-4f64-4574-864c-e2ff36bcda86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.626523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 208/424] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:26:25.629014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714e7ec-10bf-4ed2-842f-bf2a1630a979 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.629301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714e7ec-10bf-4ed2-842f-bf2a1630a979 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.629574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714e7ec-10bf-4ed2-842f-bf2a1630a979 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.635109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 209/425] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:25.635937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714e7ec-10bf-4ed2-842f-bf2a1630a979 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:25.651119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:25.673123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1de7c5-4f64-4574-864c-e2ff36bcda86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:25.674546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1de7c5-4f64-4574-864c-e2ff36bcda86 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:25.741425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714e7ec-10bf-4ed2-842f-bf2a1630a979 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:25.744274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:25.778287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:25.779437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 217/426] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:25.782666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.782666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.782666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.783791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.784070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.784354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.792822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:25.792822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:25.792822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 210/427] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:25.796334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:25.798699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:25.806793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:25.860969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:25.880883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 218/428] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:25.881886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:25.930996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dbee4-ba84-412c-98af-c574137dd67a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:25.968787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:25.969755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:25.970126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:25.970400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] PATH_INFO: `/v3/projects/141ee3a6fe3b41d5a050afe46844c163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:25.971656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] RBAC: Authorizing `identity:get_project(project_id=141ee3a6fe3b41d5a050afe46844c163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:25.973101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:25.975024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 211/429] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/projects/141ee3a6fe3b41d5a050afe46844c163 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:25.980211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.006445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd03c5-2922-46da-85d4-ed815a6b6ca9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.013788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:26.060659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.061622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.061913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.062188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.063455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:26.064926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:26.071650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 219/430] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:26.078242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.091905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a166b-e572-4a43-aa44-654d5d79d297 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.128139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.129054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.129396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.129728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.135512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:26.141427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:26.150744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:26.152538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 14:26:26.154433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:26.162776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 212/431] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:26.162776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.183698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b2ba6-7fe0-41ea-94c0-52c8ac79f114 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.219115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.220180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.220471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.220744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.222162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:26.223728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:26.225858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 220/432] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:26 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:26.226643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.262222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:26.271287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351faac-397f-4f89-9c8f-af7e5f4e2c43 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.307939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.308723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.309109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.309447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.310389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=141ee3a6fe3b41d5a050afe46844c163) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:26.310803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:26.312507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:26.319773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 213/433] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:26 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:26.320734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.372677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb3396-79ac-459e-8412-41534bd3186d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.413807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:26.430843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.431546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.431822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.432096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.434571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:26.434890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:26.436352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:26.453825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 221/434] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:26:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:26.460838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.531750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f93c1-9034-43ba-ab95-fb1988d3a548 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.578262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.579059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.579591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.580003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.581009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=141ee3a6fe3b41d5a050afe46844c163) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:26.581472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:26.583236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:26.590285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 214/435] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:26 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=141ee3a6fe3b41d5a050afe46844c163&role.id=3e228f026e734b8b9c06f578f2012163 => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:26.623432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73789955-b6a8-402b-be6f-c370336c9c88 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.665678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e120b-ef29-4dac-b7d7-3200cb0ce709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:26.666505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e120b-ef29-4dac-b7d7-3200cb0ce709 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.666826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e120b-ef29-4dac-b7d7-3200cb0ce709 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.667155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e120b-ef29-4dac-b7d7-3200cb0ce709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.731353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:26.745603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e120b-ef29-4dac-b7d7-3200cb0ce709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:26.747235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e120b-ef29-4dac-b7d7-3200cb0ce709 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:26.865098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 222/436] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:26.870360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:26.894576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:26.976985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bbb2a-b42f-42a3-b60a-0b978416d0d1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.977582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bbb2a-b42f-42a3-b60a-0b978416d0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.978053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bbb2a-b42f-42a3-b60a-0b978416d0d1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:26.979056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14c4ee-d474-434f-961d-6f95cf28c6e3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:26.980117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 223/437] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:26:26.985794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2519c-7aa7-4980-bd90-8df9dbb314bb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:26.986270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2519c-7aa7-4980-bd90-8df9dbb314bb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:26.986692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2519c-7aa7-4980-bd90-8df9dbb314bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.004895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2fe2519c-7aa7-4980-bd90-8df9dbb314bb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:27.005750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2fe2519c-7aa7-4980-bd90-8df9dbb314bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:27.019769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd77478-f529-4fc0-95ef-357d5381262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.020651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77478-f529-4fc0-95ef-357d5381262f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.020930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77478-f529-4fc0-95ef-357d5381262f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.021197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77478-f529-4fc0-95ef-357d5381262f None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.022383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77478-f529-4fc0-95ef-357d5381262f None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.023880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd77478-f529-4fc0-95ef-357d5381262f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:27.025570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 215/438] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:26 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:27.026744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645cc6-7467-4e9b-85e9-84f1dcfd85ab None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.027083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645cc6-7467-4e9b-85e9-84f1dcfd85ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.027351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69645cc6-7467-4e9b-85e9-84f1dcfd85ab None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.028759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 216/439] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:27.032200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645cc6-7467-4e9b-85e9-84f1dcfd85ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.082895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 224/440] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:27.083512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.083803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.084116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.106203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:27.106398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:27.119954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.150025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.169585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69645cc6-7467-4e9b-85e9-84f1dcfd85ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.190529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 225/441] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:27.195325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.210955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.211863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.212210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.212481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.213843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.215486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:27.217223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 217/442] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:27.218074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.326307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15478280-1b27-4885-82d9-f7dce8e910e3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.353435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda15a-cbd9-4714-9051-92e9f3410828 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.361642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.362459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.362738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.363046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.375705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:27.383907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:27.384947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:27.385757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 226/443] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:27.386390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.395024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.395823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.396100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.396364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.396480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.409620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:27.418607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:27.420214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 218/444] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:27.421705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.442061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.512272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772e228-bdc9-480d-9cd9-281327661487 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.535684 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceff308-df01-4349-b9c9-2bafdd2a86b2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.550087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.551124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.551410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.551677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.552555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=3e228f026e734b8b9c06f578f2012163, scope.project.id=92768b6befe14406aed6530166b50ea3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:27.552863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.554344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:27.560449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 227/445] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=3e228f026e734b8b9c06f578f2012163 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:27.561254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.583542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.584114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.584396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.584662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.585460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:27.585731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.587254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:27.598410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 219/446] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:27.604212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.649813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.688697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da68026-951d-4932-abf9-103d66a72af6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.715811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.727944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.728737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.729025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.729298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.730063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:27.730370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.731892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:27.746105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 228/447] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:27.747382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:27.756215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66814bd-d1e5-4010-bb10-42988615e5dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.800112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.801128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.801785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.802065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.870358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130df06-1601-465c-bb83-84bf202e5074 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:27.888495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.889958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:27.906480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.914304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49befc67-e826-45f5-9997-576403d8395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:27.915028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49befc67-e826-45f5-9997-576403d8395a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:27.915466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49befc67-e826-45f5-9997-576403d8395a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:27.915740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49befc67-e826-45f5-9997-576403d8395a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:27.984356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:27.993895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49befc67-e826-45f5-9997-576403d8395a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:27.995413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49befc67-e826-45f5-9997-576403d8395a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.005788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 220/448] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 14:26:28.011372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:28.092645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 229/449] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:28.098261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49befc67-e826-45f5-9997-576403d8395a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:28.109566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80887105-3e94-4dd0-967c-366493278e3c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:28.127272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:28.144493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:28.145353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:28.145631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:28.145897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] PATH_INFO: `/v3/projects/92768b6befe14406aed6530166b50ea3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:28.147118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] RBAC: Authorizing `identity:get_project(project_id=92768b6befe14406aed6530166b50ea3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:28.148551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.150242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 221/450] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:28 2022] GET /identity/v3/projects/92768b6befe14406aed6530166b50ea3 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:28.155918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:28.195152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49befc67-e826-45f5-9997-576403d8395a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:28.201628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:28.229651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:28.230518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:28.230814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:28.231105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] PATH_INFO: `/v3/projects/9157b1edcbb3430198b55709b5ddfe19` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:28.232332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] RBAC: Authorizing `identity:get_project(project_id=9157b1edcbb3430198b55709b5ddfe19)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:28.233826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.235547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 230/451] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:26:28 2022] GET /identity/v3/projects/9157b1edcbb3430198b55709b5ddfe19 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:28.241136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:28.254184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550db54d-476d-481c-b639-a4e6e98cffd5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:28.287479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:28.288339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:28.288626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:28.288893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] PATH_INFO: `/v3/groups/d0f965611ab245fd865708205d2b5d1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:28.290081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] RBAC: Authorizing `identity:get_group(group_id=d0f965611ab245fd865708205d2b5d1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:28.291553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.297889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 222/452] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:28 2022] GET /identity/v3/groups/d0f965611ab245fd865708205d2b5d1f => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:28.299072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:28.336433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:28.339925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca3d2-0319-4c56-8f22-ff706e1de14d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:28.375659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:28.376534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:28.376811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:28.377077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] PATH_INFO: `/v3/groups/dac9783cb06040668d7f818ca521c7d8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:28.378323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] RBAC: Authorizing `identity:get_group(group_id=dac9783cb06040668d7f818ca521c7d8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:28.380181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.381833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 231/453] 10.222.0.32 () {56 vars in 1180 bytes} [Wed May 25 14:26:28 2022] GET /identity/v3/groups/dac9783cb06040668d7f818ca521c7d8 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:28.387649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:28.404448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0281d-488f-43be-b195-cd63558a8c89 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:28.418536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:28.441663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:28.442418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:28.442774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:28.443066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:28.444045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] RBAC: Adding query filter params (group.id=d0f965611ab245fd865708205d2b5d1f, role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=92768b6befe14406aed6530166b50ea3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:28.444348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:28.445801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2322b-79af-489e-9704-164ba4f0cc74 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.451906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 223/454] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:28 2022] GET /identity/v3/role_assignments?group.id=d0f965611ab245fd865708205d2b5d1f&scope.project.id=92768b6befe14406aed6530166b50ea3&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:28.493973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903cab-2bea-40dc-a728-f5bf83d852d9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:28.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:28.531493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:28.531775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:28.532046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:28.532903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] RBAC: Adding query filter params (group.id=dac9783cb06040668d7f818ca521c7d8, role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=9157b1edcbb3430198b55709b5ddfe19) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:28.533210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:28.534685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49f1a2-2cbb-435d-8db0-944d34e64857 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:28.541575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 232/455] 10.222.0.32 () {56 vars in 1376 bytes} [Wed May 25 14:26:28 2022] GET /identity/v3/role_assignments?group.id=dac9783cb06040668d7f818ca521c7d8&scope.project.id=9157b1edcbb3430198b55709b5ddfe19&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:31.420870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1655a5-889d-4698-9d1e-fa20c97b2259 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.421919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1655a5-889d-4698-9d1e-fa20c97b2259 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.422397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1655a5-889d-4698-9d1e-fa20c97b2259 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.424303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 224/456] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:26:31.431666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3d78-4840-4aff-bcc6-3bfbb22c92e7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.432992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3d78-4840-4aff-bcc6-3bfbb22c92e7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.433755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb3d78-4840-4aff-bcc6-3bfbb22c92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.472422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5efb3d78-4840-4aff-bcc6-3bfbb22c92e7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:31.474717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5efb3d78-4840-4aff-bcc6-3bfbb22c92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:31.538666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:31.595565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 233/457] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:31.652469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1655a5-889d-4698-9d1e-fa20c97b2259 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:31.820791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1655a5-889d-4698-9d1e-fa20c97b2259 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:31.836333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce61673-2caf-4518-85ec-d411876a77f0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.836609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce61673-2caf-4518-85ec-d411876a77f0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.837042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce61673-2caf-4518-85ec-d411876a77f0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.839052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 234/458] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:31.844129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.844683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.845149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.868390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:31.869211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.869491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.869760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.871828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:31.873465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:31.881608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:31.883100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f27cf5f7-cbfd-4fcb-b5eb-4321532309b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:31.902207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:31.903317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 225/459] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf732c-47ce-465a-af78-d04bebfddc82 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf732c-47ce-465a-af78-d04bebfddc82 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.908213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf732c-47ce-465a-af78-d04bebfddc82 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.910862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 226/460] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:31.913241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a8e0-b687-4e8b-9e83-7684ad4b215d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:31.913532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a8e0-b687-4e8b-9e83-7684ad4b215d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:31.913802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a8e0-b687-4e8b-9e83-7684ad4b215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:31.933935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ebc0a8e0-b687-4e8b-9e83-7684ad4b215d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:31.935233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ebc0a8e0-b687-4e8b-9e83-7684ad4b215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:31.964778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:31.973602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 235/461] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:31.975310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.026667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 227/462] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:32.029044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2429be-1023-4147-bff6-199cf027aa9a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.029553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2429be-1023-4147-bff6-199cf027aa9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.029987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2429be-1023-4147-bff6-199cf027aa9a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.034922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 228/463] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:32.038870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2429be-1023-4147-bff6-199cf027aa9a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.102621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:32.116436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24993fe4-91a5-45c7-9cb9-96f74540a4c4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.168203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.171185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.173417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.176681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.177504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:32.177826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.179298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb856fb7-bdc4-4467-95ac-e12b1ac8743a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.202380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 236/464] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:32.202581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2429be-1023-4147-bff6-199cf027aa9a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.202712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c798-d362-449a-a194-482c37a101d0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.202712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c798-d362-449a-a194-482c37a101d0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.202712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c798-d362-449a-a194-482c37a101d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.219364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a502c798-d362-449a-a194-482c37a101d0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:32.220574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a502c798-d362-449a-a194-482c37a101d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:32.244094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.244906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.245187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.245459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.256590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.258061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:32.282279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:32.283389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:32.284716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:32.284716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:32.284716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:32.284716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:32.284716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 229/465] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:32.284716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.306693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:32.327231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 237/466] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:32.328675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c798-d362-449a-a194-482c37a101d0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.428411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39710c5c-66e8-4a67-a0d8-21b8464ef326 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.442729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:32.465359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c798-d362-449a-a194-482c37a101d0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.481430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.482810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.483379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.484344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.497956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.500184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.508521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.509213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.509524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.509835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.510614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.512344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:32.521792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:32.522627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:32.523531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:32.523531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:32.523531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:32.523531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:32.530134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 230/467] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:32.530134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.537572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 238/468] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:32 2022] POST /identity/v3/users => generated 246 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:32.538694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.617021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:32.677995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a13a2-bbe9-4942-809f-601edb8dc219 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.717795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7346576-abf2-4788-9809-48614ada3ee1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.737115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.738294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.738852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.739388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.740693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:32.741412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.743266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.764501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 231/469] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:32.764501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.766860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.767834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.768148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.768474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.777051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:32.784461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.788188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:32.808954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:32.810003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:32.812281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:32.812281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:32.812281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:32.812281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:32.812281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 239/470] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:32.812281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:32.911496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a8e5a-80cb-462f-b6b7-e2535be60ca1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.945828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:32.952251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cab99-bb3c-4ffa-a06f-2063e5af219b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:32.962458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:32.963554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:32.963985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:32.964376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:32.965657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:32.966156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:32.968381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:32.993747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 232/471] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:32.993747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:33.018513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da5f8a-cd52-40c2-8fd6-0f7ef6eedef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:33.019668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da5f8a-cd52-40c2-8fd6-0f7ef6eedef4 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.020135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da5f8a-cd52-40c2-8fd6-0f7ef6eedef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.020570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da5f8a-cd52-40c2-8fd6-0f7ef6eedef4 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.021576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da5f8a-cd52-40c2-8fd6-0f7ef6eedef4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:33.023440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da5f8a-cd52-40c2-8fd6-0f7ef6eedef4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:33.049339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 240/472] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:32 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:33.051047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbaecc-df29-4635-bdbc-2d31651452c9 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.051336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbaecc-df29-4635-bdbc-2d31651452c9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.051602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbaecc-df29-4635-bdbc-2d31651452c9 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.054806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 241/473] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:33.054806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbaecc-df29-4635-bdbc-2d31651452c9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:33.122529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:33.180688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64790708-5cc5-4e71-a8e4-beec23266ddf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:33.192944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbaecc-df29-4635-bdbc-2d31651452c9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:33.238219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:33.239168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.239601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.240007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.240889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:33.241386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:33.242956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad9510-e514-4393-9aa9-403d51762ea5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:33.246734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:33.248191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.248831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.249467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.251008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:33.254000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:33.257576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 233/474] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:33.259138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.259578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.260028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.280315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:33.281805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:33.297062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 242/475] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:33 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:33.298086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:33.348687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:33.394989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 234/476] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:33.436652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:33.509889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dd179-fa5e-40b1-9fd2-a304243c87ec None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:33.512661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:33.562773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:33.563053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.563423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.563575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.566280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:33.566456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:33.594100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 243/477] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:33 2022] POST /identity/v3/users => generated 249 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:33.620197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe9a3b-1052-47ef-b481-c9d60642d652 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:33.674353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:33.675908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.676475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.676988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.678114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:33.679892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:33.694107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:26:33.694107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 14:26:33.694107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [parameters: {'id': 'e520bd074ae84a3f8410aa0ccfb49809', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 14:26:33.694107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 25 14:26:33.720114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return ref.to_dict() May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:26:33.721203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:26:33.722401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:26:33.723417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:26:33.725557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:26:33.728178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': 'e520bd074ae84a3f8410aa0ccfb49809', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:33.730630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 14:26:33.732972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:33.737751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 235/478] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 25 14:26:33 2022] POST /identity/v3/roles => generated 144 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:33.745627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:33.885919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f1755-cc6f-47f8-a50f-f808c9d5926a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:33.945255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bc04d-1568-4cd2-a37e-b4b37cdd8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:33.946570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc04d-1568-4cd2-a37e-b4b37cdd8a25 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:33.947060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc04d-1568-4cd2-a37e-b4b37cdd8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:33.947510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bc04d-1568-4cd2-a37e-b4b37cdd8a25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:33.963916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bc04d-1568-4cd2-a37e-b4b37cdd8a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-174bc04d-1568-4cd2-a37e-b4b37cdd8a25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:33.972508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:33.973395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:33.974340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 244/479] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:26:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:33.979495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:34.031796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:34.104737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe37f85-6f1b-4d8e-b93f-d55f1c7c9622 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:34.144028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:34.144806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:34.145048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:34.145482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:34.146285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:34.146695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:34.148157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4fbcf-390e-45a8-be8d-667d30db6fa4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:34.157541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 236/480] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:35.137065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6dbae-2808-43e5-8d8c-21840e9f71dc None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:35.137597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6dbae-2808-43e5-8d8c-21840e9f71dc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:35.137597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6dbae-2808-43e5-8d8c-21840e9f71dc None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:35.139007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 245/481] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:35.152603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:35.153071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:35.153143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:35.181342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:35.182538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:35.253623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 237/482] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:35.296573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6dbae-2808-43e5-8d8c-21840e9f71dc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:35.336360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:35.548659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6dbae-2808-43e5-8d8c-21840e9f71dc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:35.632094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:35.632927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:35.633206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:35.633554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:35.649453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:35.657463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:35.658359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:35.659179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 246/483] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:35.665002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:35.735182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:35.830207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ef820-95c5-472a-af9d-eed2a6b31e8b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:35.871656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:35.872349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:35.872625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:35.872896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:35.873664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:35.873961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:35.875412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ded0f-96ce-465e-bc12-e36511d94ff7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:35.885679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 238/484] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:35.891175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.055289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.076737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e97e7c-774c-4384-92ae-c4a5fa8c2814 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.077005 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e97e7c-774c-4384-92ae-c4a5fa8c2814 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.077313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e97e7c-774c-4384-92ae-c4a5fa8c2814 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.078867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 239/485] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:36.085082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f18af-a429-44b4-80cd-818c7f203d41 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.085378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f18af-a429-44b4-80cd-818c7f203d41 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.085680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f18af-a429-44b4-80cd-818c7f203d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.098480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977b9a2-cf69-452f-b387-923230a3ac34 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.138470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d14f18af-a429-44b4-80cd-818c7f203d41 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:36.139710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d14f18af-a429-44b4-80cd-818c7f203d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:36.192247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:36.194187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.194187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.194187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.235595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:36.237121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-35546040-d8ad-419f-8b72-8e0358dbac29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:36.271419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:36.272338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:36.273613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:36.273613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:36.273613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:36.273613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:36.273613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 247/486] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:36.276485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a47e6-452b-4613-8610-3c9b3c533335 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.276485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a47e6-452b-4613-8610-3c9b3c533335 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.276485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a47e6-452b-4613-8610-3c9b3c533335 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.276485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 248/487] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:36.281283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a47e6-452b-4613-8610-3c9b3c533335 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.310975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 240/488] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:36.313294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.313782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.314254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.338391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:36.339991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:36.365579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.393763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a47e6-452b-4613-8610-3c9b3c533335 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.429407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 241/489] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:36.429407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.432493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.443099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:36.443765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.444039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.444307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.445153 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:36.445454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:36.446877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:36.463253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 249/490] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:36.464120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.566218 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1139-3cc1-4cef-beb6-826dc5d53c21 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.580243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b293557-1d22-4258-9147-59b8c9ced1a5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.606170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:36.607100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.607382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.607651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.619199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:36.619682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:36.620010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.620286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.620557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:36.627877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:36.629151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 242/491] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:36.630985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.632980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:36.640797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:36.643061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 250/492] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:36.645079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.652437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.701162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.755302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772bafa-b3fb-491d-865a-460cee826cfc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.756334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfacaf-b735-4082-8340-d9755b3499f4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.804068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:36.804068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.804068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.804068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.805403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:36.806688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:36.807386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:36.808042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:36.809688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:36.810430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:36.812543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:36.812720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:36.815797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:36.822908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 251/493] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:36.824268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:36.842561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:36.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 243/494] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:36.846066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:36.936279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.953958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979431b6-e19b-4584-a06c-0ae304d3c2a0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:36.968128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:36.973379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626ea9f-f998-4fed-b1ae-4a1bf4a2ca20 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.000334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.001027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.001305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.001580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.002348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:37.002657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.004119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc1785-2817-4f20-9b07-48ebd5a8ee77 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.014611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 252/495] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:37.015998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3bc57-5d53-4f6d-8a92-8ecb2e81fd5f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.016287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3bc57-5d53-4f6d-8a92-8ecb2e81fd5f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.016558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3bc57-5d53-4f6d-8a92-8ecb2e81fd5f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.022006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 253/496] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:37.022006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3bc57-5d53-4f6d-8a92-8ecb2e81fd5f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.025131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.026247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.026708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.027255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.039656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.041550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:37.061944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:37.062908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:37.063770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:37.063770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:37.063770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:37.063770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:37.066822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 244/497] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:37.066822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.147518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3bc57-5d53-4f6d-8a92-8ecb2e81fd5f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.174403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adb291-0533-4b0b-b7ad-eb316b29bd07 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.186640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.187354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.187635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.187907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.188761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:37.189069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.190525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b1071-3f6c-4611-a9df-604cf132cf5d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.198881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.211076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.211875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.212145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.212409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.223464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.225106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.228187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 254/498] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:37.229516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d2586-9783-4592-a1e9-dc64f641ae25 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.229799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d2586-9783-4592-a1e9-dc64f641ae25 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.230073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d2586-9783-4592-a1e9-dc64f641ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:37.243331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:37.244243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:37.245058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:37.245058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:37.245058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:37.245058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:37.245058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 245/499] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:37.245712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.248386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b26d2586-9783-4592-a1e9-dc64f641ae25 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:37.249773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b26d2586-9783-4592-a1e9-dc64f641ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:37.278809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.339671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 255/500] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:37.341845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8f67d-5234-4d27-8a4a-09dacf854901 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.342311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8f67d-5234-4d27-8a4a-09dacf854901 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.342791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8f67d-5234-4d27-8a4a-09dacf854901 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.345406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 256/501] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:37.346383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8f67d-5234-4d27-8a4a-09dacf854901 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.372262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce13eb-9442-4f05-9135-0bf04adb11f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.411800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.412377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.412654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.412928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.413931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:37.413931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.415352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.427906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 246/502] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:37.428800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.462650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8f67d-5234-4d27-8a4a-09dacf854901 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.464168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.496768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.497426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.497699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.497967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.528922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef951d6-c439-4c44-b429-ccb0bef817a7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.530946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.573858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.575691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.576560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.577239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.577504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.577780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.578502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:37.578833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.580198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853325e8-14bf-4c4a-b4d1-5a641beabda2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.591797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 247/503] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:37.593149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.593535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.593702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.610522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:37.611791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:37.679683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 257/504] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:37.680466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.690949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 248/505] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:37.690949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.716567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.749894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.789443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b618ad-9df4-4354-b04e-881a61c42c72 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.793454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0a39f-3a4c-440f-b394-a805a4ef12c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.825186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.825999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.826277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.826550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.829012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:37.829824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:37.830103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:37.830519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:37.837021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:37.837960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:37.839804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:37.845007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:37.845918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:37.846696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 258/506] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:37.847387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:37.857363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:37.858494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:37.859412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 249/507] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:37.859958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:37.948916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:37.961766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a7f99-e2f7-4a92-a1da-8a51edae3b60 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.966890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac31a-f72a-4faf-84b2-cb1edf5e2591 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:37.987617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.001172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.001973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.002253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.002524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.005660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.006462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.006738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.007017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.010694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.012215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.017986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.019484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:38.035355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:38.037106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 259/508] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:38.038900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:38.040057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:38.041117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:38.044792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:38.044792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:38.044792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:38.044792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:38.044792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 250/509] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:38.044792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.154459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b35360d-f3f2-41d0-85e5-1ed8d25c4796 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.166380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fdaab-8c60-47fb-a352-272a2580734b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.194476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.195344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.195609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.195879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.205305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.205897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.206179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.206451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.207113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.207213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:38.207530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.208616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.208890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.214659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.219280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 251/510] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:38.221053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:38.233871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:38.235104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:38.235934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:38.235934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:38.235934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:38.235934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:38.235934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 260/511] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:38.235934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.274827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.349872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85bd7f-74c5-46e2-8c39-4686adfd0429 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.357451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b740d-1750-49fb-a98d-e5275f416bb4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.386501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.387265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.387547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.387818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.388672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:38.388979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.390415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.395418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.396101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.396381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.396652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.397506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:38.397814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.399309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.417375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 261/512] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:38.418278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.432750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 252/513] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:38.433711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.494847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.510529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.519913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473b239-674e-4a18-af51-58a43cc375d1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.537267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828967e7-18f7-4f83-863a-6f20ffdc35f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.555226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.555963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.556240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.556516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.557247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:38.557551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.559047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.570826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 262/514] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:38.571269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.577021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.577834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.578118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.578422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.588847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.590372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:38.608545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:38.609627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:38.610511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:38.610511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:38.610511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:38.610511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:38.614460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 253/515] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:38.615219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.682926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e320452-fe76-4e9e-b65f-de9e25c7d87e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.724951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.725403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914dce5-c3c0-4cd0-880c-abd0c4aa477c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.725629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.725907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.726181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.726950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:38.727249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.728653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.740603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 263/516] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:38.741204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.741803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.761817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.777968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.779039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.779325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.782272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.865673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7e56b-a797-4be4-9f18-3e334140eb85 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:38.865911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.868335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.903226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:38.903881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:38.904162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:38.904430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:38.974593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 254/517] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:38.975478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:38.977020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:38.978529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:38.987204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:38.993776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.076953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d07a87-dd3f-4a01-b459-4607d2aa08cf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.086339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 264/518] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:39.087283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.113440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.114243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.114522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.114860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.122166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.123647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:39.138023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:39.139118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:39.139954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 255/519] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:39.140428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.197749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d6fff-3d1d-4667-86f2-47fbc99c518b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.210927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.213367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.236156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.236895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.237193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.237463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.238199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:39.238501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.239905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.244704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12eab8-b28f-44ba-925f-a80cf568fc88 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.251574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 265/520] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:39.252477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.282659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.283369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.283656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.283959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.284669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.286341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.309710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 256/521] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:26:39 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:26:39.310562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.406842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345d6b-4edf-4459-a531-f4b66c03f3b9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.448676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.449498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.449774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.450041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.462535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.464082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.467335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2658e26-dc32-4971-b69c-69b6c91feea1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:39.482641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:39.483736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:39.484731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:39.484731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:39.484731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:39.484731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:39.484731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 266/522] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:39.484731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.505704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.518351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.519227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.519486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.519732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.519832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.533753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.534935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:39.555140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:39.556048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:39.556861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:39.556861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:39.556861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:39.556861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:39.559020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 257/523] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:39.559877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.612164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154417e0-626d-443f-904e-aeda161a7719 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.647338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.647993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.648292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.648549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.649371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:39.649673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.651184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.670307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 267/524] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:39.671198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.713881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8fd0b-535a-4d49-b35c-e0699f9aefbb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.752280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.753451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.753865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.754300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.763401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.764180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.765798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:39.780926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:39.781992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:26:39.783075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:39.784987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 258/525] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:39.786039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.787245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843b60c-554d-4e73-a5e7-425748254c69 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.828698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.829466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.829748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.830019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.830780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:39.831076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.832570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.833469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:39.850614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 268/526] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:39.851469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.899335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52bbd0-fc3a-4267-b75e-b23e4a02d1ec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.934672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.935303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.935579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.936186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.936571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:39.936876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:39.938258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:39.954530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 259/527] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:39.959557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:39.962057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483270f-1b67-46b4-a872-d62c0ddbd84b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:39.997796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:39.998456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:39.998730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:39.999028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:39.999871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:40.000180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.001622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.007726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 269/528] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/role_assignments?user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:40.008609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.021406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.099872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.106070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3fa18-84bb-43e0-974c-15ce515cb9bc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.131712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eb93a-bf3b-4eaf-b912-5d3e248b9d75 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.153579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.155014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.155509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.156145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.157621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:40.158159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.160590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.177282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.178128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.178406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.178726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.187164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.188014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 260/529] 10.222.0.32 () {56 vars in 1139 bytes} [Wed May 25 14:26:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:40.188584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.189319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:40.205913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:40.206859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:40.207688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:40.212057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 270/530] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:40.213058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.286730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.325935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468210c6-05e2-4e88-941c-064dc082b440 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.339231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618b05-2966-4a50-935c-e70578c3436a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.362006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.362679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.362959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.363228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.393397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.394576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.395153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.395679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.405989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.408560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:40.429797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:40.430699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:40.431574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:40.432673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 261/531] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:40.433865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.459941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.461461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.462941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.548470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.582921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc395e-b915-43fb-9a31-cef20a7fd5b9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.594833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 271/532] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:40.596059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.623223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.625057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.625911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.626730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.628833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:40.629614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.632773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.653588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 262/533] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:40.654954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.705827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57584837-b951-4261-bd7c-bcfb38609dbd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.743448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.744146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.744424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.744694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.745509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:40.745978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.748844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.750461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.765375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 272/534] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:40.766576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.777042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd452b2-d863-4557-90e3-1c45ebbe6c40 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.798055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:40.826155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.827345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.827897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.828361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.839873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.841872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:40.861198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:40.862125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:40.863056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:40.863056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:40.863056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:40.863056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:40.865562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 263/535] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:40.866806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.878586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc505d12-b004-4777-a338-cf86d8d077e6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:40.915724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:40.916390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:40.916670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:40.916951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:40.917967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=175dd1ccd6104b18aea8e69bb5ec036a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:40.918280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:40.919820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:40.927330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 273/536] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/role_assignments?user.id=175dd1ccd6104b18aea8e69bb5ec036a&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:40.928361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:40.983338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d759ca-6893-4554-be06-30162ed37217 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:41.015386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:41.024225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:41.025047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.025424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.025894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=cc3c334dc46b47a3b8e415e913c29249) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:41.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:41.029065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:41.036089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 264/537] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/role_assignments?user.id=cc3c334dc46b47a3b8e415e913c29249&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:41.044630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:41.052067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e42b7d-b161-4839-9799-459d04d1433b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:41.097002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:41.098058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.098539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.098995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.099922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:41.100376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:41.101994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:41.116772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 274/538] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:41.122302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:41.268701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a67b5-a05d-40b1-b772-f7e57a231722 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:41.311239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:41.312032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.312306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.312572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.319477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:41.321053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:41.335855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:41.336726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:41.338045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 265/539] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:41.343079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:41.452569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb4b6c-70b9-45f3-b51b-02e5968c0292 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:41.488750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:41.489416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.489691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.489958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.490774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:41.491076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:41.492529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477d042-dc98-4499-9858-78a293dc557a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:41.504740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 275/540] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:41.510486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:41.585753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:41.657465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dbbb0-7c80-42b8-882d-a63111c79219 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:41.705804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:41.706514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.706815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.707083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.707965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=db335e266cd14d45868e6875a9ea34d4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:41.708271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:41.709702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62e538-d33c-49a7-902e-03c8df80b7b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:41.718721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 266/541] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:41 2022] GET /identity/v3/role_assignments?user.id=db335e266cd14d45868e6875a9ea34d4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:41.748312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99503c0e-6152-4c99-9c9e-365da8974de2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.748634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99503c0e-6152-4c99-9c9e-365da8974de2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.748864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99503c0e-6152-4c99-9c9e-365da8974de2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.750399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 276/542] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:41.756504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:41.757128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:41.757600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:41.789245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:41.790973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:41.914631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 267/543] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:41.995096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99503c0e-6152-4c99-9c9e-365da8974de2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:42.014684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:42.140567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99503c0e-6152-4c99-9c9e-365da8974de2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:42.190052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:42.190816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:42.191096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:42.191365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:42.211687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:42.219485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:42.220318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:42.221526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 277/544] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:42.230172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:42.313292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:42.399445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089dcc0-b8b3-43c7-87ae-a4937254d2ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:42.437217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:42.437906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:42.438185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:42.438453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:42.439269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:42.439570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:42.441007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:42.450190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 268/545] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:42.455864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:42.566937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:42.606640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab05af8-030c-4894-b017-aadea791287c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:42.653370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:42.654194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:42.654472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:42.654743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:42.677298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:42.678827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7e99d4f6-83ad-4960-be86-8f106a2066d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:42.710315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:42.711226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:42.712035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:42.712035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:42.712035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:42.712035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:42.712035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 278/546] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:42.719871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:42.796755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b97ce7-e63a-4ee2-9fe2-12f04c048b97 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:42.811382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b97ce7-e63a-4ee2-9fe2-12f04c048b97 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:42.811382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b97ce7-e63a-4ee2-9fe2-12f04c048b97 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:42.811382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 279/547] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:42.811382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:42.811382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:42.811382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:42.851947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:42.853566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:42.875430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27173942-8ba9-4080-9d7e-a8a506ae5957 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:42.907141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:42.918122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:42.919439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:42.920955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:42.921438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:42.923266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:42.925326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:42.928247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:42.943402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 269/548] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:42.949422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.017620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 280/549] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:43.054532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.112267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:43.119803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33db9-b470-4b0d-a208-ce7003c8bf80 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.161518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeb765-9076-448e-823b-184ecb5d7a3d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.184327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.185146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.185423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.185695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.198318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:43.203099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.204217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.204703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.205170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.207026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:43.217748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:43.224667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:43.225766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:43.226680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 270/550] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:43.227836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:43.228680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:43.229644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 281/551] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:43.236370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.241234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.290835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:43.434618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefd4ef-af97-4231-9a01-91649dfcafc3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.450320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea18e3-e316-4053-ba8e-5ec7cde314f3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.478944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:43.506343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.508124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.509171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.509635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.510878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:43.512348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:43.514474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:43.519374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.520357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.520920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.521399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.523022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:43.525008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:43.526611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:43.543655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 282/552] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:43.549171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.554873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 271/553] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:43.559785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.693305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f8510-a9a9-442c-bc61-7baa4b948704 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.710299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:43.732653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.733481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.733764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.734033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.734462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f664b2c-3320-48c8-8139-b0e9cb30e213 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.746873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:43.748421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3a8e2b0a-f7c8-4487-bf26-23d12a27f2c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:43.768497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:43.771819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:43.772654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:43.772654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:43.772654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:43.772654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:43.772654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 283/554] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:43.772950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.772950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.773099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373262-35d9-49e6-8c04-87bc20f302f1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.773099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373262-35d9-49e6-8c04-87bc20f302f1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.773099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373262-35d9-49e6-8c04-87bc20f302f1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.773793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 284/555] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:43.779899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373262-35d9-49e6-8c04-87bc20f302f1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.784178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.784576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.820389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:43.862829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:43.864294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:43.910959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373262-35d9-49e6-8c04-87bc20f302f1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:43.954941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:43.955689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.956071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.956352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.957151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:43.957460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:43.959005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e86eb-a3b1-429f-993e-925e730c7faa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:43.959694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:43.974765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 285/556] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:43.976156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:43.976448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:43.976723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:43.981193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 272/557] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:43.982095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:43.999825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:44.001276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:44.076561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 286/558] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:44.077402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.088061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847180-9e2c-4078-b9ae-7af662fe733b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.096717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.134470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.135304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.135581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.135878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.143148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.144592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0e350203-5ef6-433a-aae5-4760ab94c1db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:44.158927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:44.160406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:44.161350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 273/559] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:26:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:44.161943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388ac5c-c8a6-4d53-9b59-9c1b198d76a6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.161943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388ac5c-c8a6-4d53-9b59-9c1b198d76a6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.162076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388ac5c-c8a6-4d53-9b59-9c1b198d76a6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.166862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 274/560] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:44.166862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388ac5c-c8a6-4d53-9b59-9c1b198d76a6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.200899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44492e-3ba5-4242-ae68-79f74d64a88c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.229907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.238476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.239323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.239605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.239876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.250818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.252293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:44.270012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:44.271222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:44.272285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:44.272285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:44.272285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:44.272285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:44.275761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 287/561] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:44.276671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.294362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388ac5c-c8a6-4d53-9b59-9c1b198d76a6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.329629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf745608-1157-43ec-a64b-1910d9566f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.330342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf745608-1157-43ec-a64b-1910d9566f42 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.330615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf745608-1157-43ec-a64b-1910d9566f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.330909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf745608-1157-43ec-a64b-1910d9566f42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.341958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf745608-1157-43ec-a64b-1910d9566f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bf745608-1157-43ec-a64b-1910d9566f42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:44.349609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:44.350535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:44.351937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 275/562] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:44.352735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.352735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.352735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.363240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.370407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:44.371717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:44.394897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64965b26-ff3e-47cb-a0f7-7851d9275142 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.427623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.428298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.428595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.428850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.429703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:44.430016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.431636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.438934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 276/563] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:44.439787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.451471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 288/564] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:44.452305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.478642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.541851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf77f6-769f-4152-90cc-6b4af0d6f16a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.553262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c39e80-4368-4b14-9a0a-962a849a19e2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.574417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.575103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.575376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.575639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.576361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:44.576656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.578034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e41a8e-3db7-4c8e-aa8b-41e3772a61e7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.579880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.588340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.589010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.589283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 277/565] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:44.589444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.589646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.590397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:44.590572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396cfbb-9203-4911-b137-d6caff3a46d6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.590671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.590882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396cfbb-9203-4911-b137-d6caff3a46d6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.591153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396cfbb-9203-4911-b137-d6caff3a46d6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.592125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.594791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 278/566] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:44.594791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396cfbb-9203-4911-b137-d6caff3a46d6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.601715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 289/567] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:44.602527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.695841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.700891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce68238-85a9-48e4-bf1e-e4b00518bc3b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.703457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396cfbb-9203-4911-b137-d6caff3a46d6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.734765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.735599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.735877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.736151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.737500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.738304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.738576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.738865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.746723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:44.749732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.751228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:44.754483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:44.755477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:44.756282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:44.759443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 290/568] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:44.760323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-23ca468f-7c3e-4b5d-832a-b9fe57b91068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:44.768670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:44.769558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:44.770419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:44.770419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:44.770419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:44.770419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:44.770419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 279/569] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:44.771295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.771582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.771833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.789666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:44.790920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:44.814514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.861252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 280/570] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:44.862205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.865261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633221b-458c-4a05-8b0f-80d97cf0d0f3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:44.899414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:44.900228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:44.900512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:44.900787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:44.907782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:44.909262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:44.925765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:44.926972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 291/571] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:44.927928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:44.964889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:44.990408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719a6b-c244-4cf5-b6de-64e784f2f8a6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.031068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.031871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.032144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.032413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.041690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de361c79-a157-4cb7-b92c-cb830653ed24 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.043563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.045053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.054038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:45.063906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:45.065039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:45.066037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:45.066037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:45.066037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:45.066037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:45.066037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 281/572] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:45.066037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.096032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.096712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.096989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.097255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.098009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.098311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.099766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.116262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 292/573] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.117055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.195237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887e995-aa23-46e0-b0e9-745d1cf5b99f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.229172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.230901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62b64-696e-4318-9eff-530a1328b89a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.235535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.236204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.236479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.236747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.237613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.237901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.239455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.251163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 282/574] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.252031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.267581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.268546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.268833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.269106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.280494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:45.288353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:45.289463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:45.290446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 293/575] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:45.291277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.324384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.364733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe042401-241a-4a91-b6d2-1c7a80a1918e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.394218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06935233-cb88-47b4-8897-5cc5bf0ce1fd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.398961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.399640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.399916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.400185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.400989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.401303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.402910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.415696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 283/576] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.416661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.429430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.430097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.430375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.430645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.431413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.431717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.433158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.445556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 294/577] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.446713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.472962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.524639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a1b62-612c-4f6e-afaf-4209bb572fb4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.536475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.546715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7ec20-6588-4534-be9a-3e52ad6af9d2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.566887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.567634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.567908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.568253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.580361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.581085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.588909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.589768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.590049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.590325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.598047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.599540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:45.601233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:45.602864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:45.603820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:45.603820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:45.603820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:45.603820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:45.603820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 284/578] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:45.603820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:45.617863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:45.619176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 295/579] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:45.620381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.710233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab0f6c-5883-432f-a484-4d8c4bbf93dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.711727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.728248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad8d9a-fe7d-43a4-8ba1-85dc4fe5d69c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.746155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.747013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.747213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.747364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.748099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.748395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.749757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.757779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 285/580] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.758586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.768738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563512-488c-43d6-a621-455865b1d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.769404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563512-488c-43d6-a621-455865b1d2df None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.769687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563512-488c-43d6-a621-455865b1d2df None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.769924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563512-488c-43d6-a621-455865b1d2df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.770931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563512-488c-43d6-a621-455865b1d2df None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=5808b2b191a14c358ec62171be87a01f) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.771141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563512-488c-43d6-a621-455865b1d2df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.772604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563512-488c-43d6-a621-455865b1d2df None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.778223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 296/581] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/role_assignments?user.id=5808b2b191a14c358ec62171be87a01f&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.779046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563512-488c-43d6-a621-455865b1d2df None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.800073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:45.897539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821593-af4b-484d-b9b6-0965984d971a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.908620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563512-488c-43d6-a621-455865b1d2df None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:45.943507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.944659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.945087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.945520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.948693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:45.949705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:45.949996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:45.950276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:45.951065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:45.951376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.952797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.955753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:45.957245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:45.966696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 297/582] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:45.968023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:45.972942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:45.974109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:26:45.975345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:45.979628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 286/583] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:45.979948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.010364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.058344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.082485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2aba0-8820-4b61-bcf5-d9bccf97d2bf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.097108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d285898-14e1-4c91-99d0-4248e7659fd9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.120874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.121510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.121793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.122062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.122928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:46.123210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.124661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.141530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 298/584] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:46.141762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.141874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.141960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.141960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.141960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.157073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.158960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:46.178785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:46.179712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:46.180568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:46.180568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:46.180568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:46.180568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:46.180568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 287/585] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:46.180850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.256003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ad44f-9b26-4929-9154-cf43e8348947 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.265148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.291856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964bbd-904d-4109-bc52-564bf3dfb70b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.294449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.295339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.295613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.295880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.300070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.305927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.307725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:46.323576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:46.324859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:26:46.326009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:46.326685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 299/586] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:46.327410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.338223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.338944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.339225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.339529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.340537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:46.340844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.342301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.362672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 288/587] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:46.363278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.442895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8a8c1-c256-4b09-8596-e65e3a735e9f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.475246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf36ae-f2b6-4568-9511-4380872acd2b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.481554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.482503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.483140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.483261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/eeee4e4e0fc74d7db8b5cb9d0ba806a4/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.498721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=eeee4e4e0fc74d7db8b5cb9d0ba806a4, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.500234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.511572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.512541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.513040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.513501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.514424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:46.514538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 300/588] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:46 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/eeee4e4e0fc74d7db8b5cb9d0ba806a4/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:46.515055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.515323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.516957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.518895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.535098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 289/589] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:46.536282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.584566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.650622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0b118-0adf-49a8-bfc2-1a6504540926 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.675673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78c23d-a2c6-4747-9fb3-b065ba55fe41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.690836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.691842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.692294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.692766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.693803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:46.694283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.696018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.718845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 301/590] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:46.719888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.719995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.721075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.721520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.721966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.736771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.739117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:46.757116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:46.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:46.759671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:46.759671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:46.759671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:46.759671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:46.761525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 290/591] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:46.762501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.824167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.838059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce4bed-141a-48bf-93f7-1cc0b90c1a9b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.841269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:46.877012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.878212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.878630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.879098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.887454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728208c8-5d58-42b2-8370-b764223e2912 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:46.888056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.890187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:46.907970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:46.909223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:26:46.910273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:46.913690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 302/592] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:46.914972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:46.928575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:46.929488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:46.929772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:46.930038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:46.944943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:46.946433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:46.966085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:46.967077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:46.967925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:46.967925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:46.967925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:46.967925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:46.967925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 291/593] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:46.968632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.047310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95995d-da27-4c0f-ae58-ce7976457159 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.097080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.098008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.098736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.099102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.100111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:47.100422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.101171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.103211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.105795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24460c78-b9e6-406e-a97e-51732b6408a4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.120498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 303/594] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:47.121683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.152251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.161000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.162149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.162638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.163305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.164400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:47.164913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.166597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.187671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 292/595] 10.222.0.32 () {56 vars in 1139 bytes} [Wed May 25 14:26:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:47.188956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.241795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b800b8-6514-470f-83cf-35f0a1a23cc1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.278008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.278681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.278986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.279411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.280267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:47.280536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.282027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.295539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 304/596] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:47.296478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.298602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac61c9-e660-4c30-b210-ccc2ee42c193 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.337199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.338488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.338997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.339618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.347690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.349646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:47.365522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:47.366473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:47.367381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:47.370424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 293/597] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:47.372039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.389220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.399934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.410161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f0118-9964-4d85-b4c1-ba004541792d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.447352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.448258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.448545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.448879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.460248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.461712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:47.481468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:47.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:47.484074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:47.484074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:47.484074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:47.484074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:47.484074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 305/598] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:47.484356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.499953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c5025-a211-42fa-b2d3-8205bf16a087 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.536872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.538151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.538632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.539159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.547197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.548993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:47.565074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:47.566027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:47.566917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:47.569570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 294/599] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:47.570787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.590994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9bcc1-eace-42d8-837f-8a970470eee3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.626420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.627335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.627616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.627880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.628678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:47.628982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.630442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.633353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.653319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 306/600] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:47.653319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.717994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.758355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738ba3a-5fc6-42f2-bf21-a31ceda20d39 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.761017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d1e9b-bb7b-4c68-9619-f153477a0785 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.803207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.803792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.804081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.804389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.805192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:47.805506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.806989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.826795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 307/601] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:47.826795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.839440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.841253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.842091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.843086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.844880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:47.845813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:47.849013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:47.879174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 295/602] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:47.881405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:47.885039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:47.943127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db85c8d-1590-4c82-add1-152268e5f922 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:47.981756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:47.982790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:47.983066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:47.983338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/cc3c334dc46b47a3b8e415e913c29249/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:47.999372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=cc3c334dc46b47a3b8e415e913c29249, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:48.001141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:48.011384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151268a-8578-4cf9-aecd-e4ae434579f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:48.022788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 308/603] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:47 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/cc3c334dc46b47a3b8e415e913c29249/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:48.022788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:48.065120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaab2a9-f052-430a-88ed-6c02dec47086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:48.065931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab2a9-f052-430a-88ed-6c02dec47086 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.066204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab2a9-f052-430a-88ed-6c02dec47086 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.066608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaab2a9-f052-430a-88ed-6c02dec47086 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/175dd1ccd6104b18aea8e69bb5ec036a/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.071122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:48.085805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab2a9-f052-430a-88ed-6c02dec47086 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=175dd1ccd6104b18aea8e69bb5ec036a, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:48.087327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaab2a9-f052-430a-88ed-6c02dec47086 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:48.112630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 296/604] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:47 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/175dd1ccd6104b18aea8e69bb5ec036a/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:48.114237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6fe70-9fb1-4691-a948-a7937315c3ad None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.114517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6fe70-9fb1-4691-a948-a7937315c3ad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.114981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6fe70-9fb1-4691-a948-a7937315c3ad None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.116390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 297/605] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:48.122673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.123050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.123287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.144389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:48.145923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:48.184409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff1450-d18f-4e08-932a-93ce3f7babdb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:48.205274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:48.239138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:48.240346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.240938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.241412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.250400 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:48.252327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:48.268903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:48.269849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:48.270787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:48.273628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 309/606] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:48.284059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:48.320675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 298/607] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:48.343784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:48.444327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac51ed-a311-484c-9d9e-2ba779646088 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:48.484657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:48.485809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.486309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.486953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.488183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:48.488720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:48.490539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:48.495833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c345d-6927-4901-b6fd-791650d7ce3e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:48.515036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:48.516890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 310/608] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:48.533276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:48.539318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:48.540143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.540424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.540698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.563196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:48.571434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:48.572376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:48.573279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:48.574994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 299/609] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:48.586371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:48.601785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:48.698368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baca5b6-fae7-4a61-aa2f-75bc194df098 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:48.754328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:48.755281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.755561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.755836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/db335e266cd14d45868e6875a9ea34d4/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.788421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=db335e266cd14d45868e6875a9ea34d4, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:48.790323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:48.793782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192fa55-de66-45ec-a45d-df5bb8103051 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:48.817409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 311/610] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:48 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/db335e266cd14d45868e6875a9ea34d4/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:48.835213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:48.835861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:48.836141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:48.836418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:48.837239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:48.837548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:48.839216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc8355-6f5f-4120-8187-04712f3f8d3d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:48.850712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 300/611] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:48.860268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:48.999140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:49.028584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8ebf5-a868-4f08-9be4-bd8d27cd872b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.028955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8ebf5-a868-4f08-9be4-bd8d27cd872b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.029146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8ebf5-a868-4f08-9be4-bd8d27cd872b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.030906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 301/612] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:26:49.039810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.040122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.040359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.041140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ef8b2-99d9-4522-b71c-eb06e0c50fbb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.062952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:49.064170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:49.092309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:49.093142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.093419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.093690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.109077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:49.112321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:49.113866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:49.136888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:49.138722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:49.139979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:49.139979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:49.139979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:49.139979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:49.139979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 312/613] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:49.145201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:49.154969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 302/614] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:49.189381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:49.258179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e7630-a4f6-4a76-b5e2-d0f00359da2a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.292083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:49.300877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:49.301809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.302093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.302366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.303204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:49.303510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:49.305013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:49.323218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 313/615] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:49.331623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:49.426874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc7fe5-e0e0-446d-bb82-32d767989780 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.498785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:49.499613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.499890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.500157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.521906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:49.529861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:49.530817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:49.531662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:49.534118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b977-f31b-41e0-9673-3c1ed88ad96a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.537827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 303/616] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:49.539226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:49.598603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:49.604635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:49.606636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.607104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.608117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.623738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:49.627176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:49.646517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:49.647720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:26:49.648856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:49.660133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 314/617] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:49.667876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:49.757025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc3678-301a-47cc-b604-8d5c27c85206 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.776496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:49.805691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:49.807415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.807884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.808334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:49.809247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:49.809705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:49.811444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:49.822207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 304/618] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:49.828129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:49.914512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9687e-14ce-46a2-a110-fbdf57d0df0a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.943008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:49.954740 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b20e0-8b36-4ab4-9d72-b611f18486bb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:49.991417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:49.992792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:49.993636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:49.994114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.006588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.008807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:50.027370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:50.028313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:50.029157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:50.029157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:50.029157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:50.029157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:50.031571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 305/619] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:50.037201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.038715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.039565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.039849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.040120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.041176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:50.041284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.043155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.089572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 315/620] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 25 14:26:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:50.100984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.157760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f6bda-64a7-4448-8177-59c0ed4331af None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:50.194645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.196515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.197226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.197915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.199247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:50.199906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.201493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.215289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 306/621] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:50.221089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.238194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:50.359887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22e12-9895-4d2d-916c-b642de11dd2b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:50.374771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:50.379933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d184-cc61-4cee-94ae-64c562265423 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:50.418489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.419742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.420236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.420716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.429116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.430970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.439526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.440353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.440636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.440917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:50.447440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:50.448358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:50.449317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:50.451921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 307/622] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:50.457798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.460748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.464530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:50.500591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:50.501536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:50.502392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:50.502392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:50.502392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:50.502392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:50.502392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 316/623] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:50.512366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.557734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:50.597219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b024c-4206-49f6-8863-5c1be1af7fb8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:50.632981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.634155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.634572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.635033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.635616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe68946-0817-49cf-824d-de50f6ae11cc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:50.636217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:50.636675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.639239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a07b9-517b-41c9-af69-363463e91f0b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.667617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 308/624] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:50.667617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a02d8-9a3e-409f-b082-cb898a8dc4f6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.667617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a02d8-9a3e-409f-b082-cb898a8dc4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.667617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a02d8-9a3e-409f-b082-cb898a8dc4f6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.670878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 309/625] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:50.671729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a02d8-9a3e-409f-b082-cb898a8dc4f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.683777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.685187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.685888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.686356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.687552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:50.688028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.690256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd97c7-ea12-43e3-ba16-9da1b56a7e62 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.705975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 317/626] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:50.707772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.708213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.708665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.729985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:50.732328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:50.786628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:50.807510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a02d8-9a3e-409f-b082-cb898a8dc4f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:50.831944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:50.846472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 318/627] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:50.846472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:50.857980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d640-a1a5-46e5-bb3d-73d762419ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:50.858686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d640-a1a5-46e5-bb3d-73d762419ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:50.858997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d640-a1a5-46e5-bb3d-73d762419ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:50.859320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d640-a1a5-46e5-bb3d-73d762419ea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:50.939082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d640-a1a5-46e5-bb3d-73d762419ea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:50.940384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d640-a1a5-46e5-bb3d-73d762419ea0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:50.988927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196355a-2fcf-430d-a558-1abb51b3ea29 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.030525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.031365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.031638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.031906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.041381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.042935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:51.063636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:51.064577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:51.065521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 319/628] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:51.066003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.078608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 310/629] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:51.078608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbd1e-a5f0-49fe-b24d-d6128f043f1e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.078608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbd1e-a5f0-49fe-b24d-d6128f043f1e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.078608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabbd1e-a5f0-49fe-b24d-d6128f043f1e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.078608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 311/630] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:51.078608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbd1e-a5f0-49fe-b24d-d6128f043f1e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.083592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.100861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.180711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a38bf3-4f0b-4cf8-a58e-65145bdc68a1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.186006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabbd1e-a5f0-49fe-b24d-d6128f043f1e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.223547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bb447-c41d-4345-ba5d-d7fcc82b26f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.224314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb447-c41d-4345-ba5d-d7fcc82b26f4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.224542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb447-c41d-4345-ba5d-d7fcc82b26f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.224802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb447-c41d-4345-ba5d-d7fcc82b26f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.226725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.227427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.227710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.227982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.228781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:51.229084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.230564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:51.238352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bb447-c41d-4345-ba5d-d7fcc82b26f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:51.242801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 312/631] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:51.243638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-508bb447-c41d-4345-ba5d-d7fcc82b26f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:51.248413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:51.249314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:51.250131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:51.257003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 320/632] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:51.257003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.257003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.257003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.270053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:51.271308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:51.341831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.346051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.358774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 321/633] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:51.358774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.381216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fce6f-7802-424b-86f1-ada5b20ada06 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.413866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.414657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.414962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.415228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.426484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.427952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:51.444695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:51.445664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:51.446549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:51.446549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:51.446549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:51.446549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:51.446549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 313/634] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:51.446996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.462340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adb2cb-d995-45f1-b9d6-191a4fef906a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.496710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28285b-a188-4980-aa0a-d065804720b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.497616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28285b-a188-4980-aa0a-d065804720b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.497888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28285b-a188-4980-aa0a-d065804720b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.498138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28285b-a188-4980-aa0a-d065804720b4 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/5808b2b191a14c358ec62171be87a01f/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.512529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28285b-a188-4980-aa0a-d065804720b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=5808b2b191a14c358ec62171be87a01f, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.513983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28285b-a188-4980-aa0a-d065804720b4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:51.527511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 322/635] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:26:51 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/5808b2b191a14c358ec62171be87a01f/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:26:51.528796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e6651-e3b1-4ba5-835e-4a5039b91d28 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.529077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e6651-e3b1-4ba5-835e-4a5039b91d28 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.529347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e6651-e3b1-4ba5-835e-4a5039b91d28 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.530607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 323/636] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:51.531337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e6651-e3b1-4ba5-835e-4a5039b91d28 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.570254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff31438-73a7-475f-b54e-30ad272f424b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.589059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.606930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.613120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.614441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.614441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.614441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.615426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:51.615737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.617145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:51.627529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 314/637] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:51.628493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.658129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e6651-e3b1-4ba5-835e-4a5039b91d28 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.700866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78550d34-37ca-4388-9ade-46bd31242601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.701685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550d34-37ca-4388-9ade-46bd31242601 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.701960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550d34-37ca-4388-9ade-46bd31242601 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.702283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78550d34-37ca-4388-9ade-46bd31242601 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.716402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78550d34-37ca-4388-9ade-46bd31242601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-78550d34-37ca-4388-9ade-46bd31242601 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:51.729335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:51.730421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:51.731618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 324/638] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:51.733450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.733450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.733618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.751355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:51.752481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:51.755799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5cb2-fb83-46b0-b8d3-42e10ea1ff2b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.796047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.796663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.796948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.797220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.797956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:51.798261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.800678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:51.820493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 315/639] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:51.820493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.838885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 325/640] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:51.840555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:51.883496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.895520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:51.947050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97251b7d-0a3c-4d11-ac42-991c77a232fc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.948807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4cb6-eef0-48d8-b70e-4e66b5eed0b2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:51.984644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.985711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.986133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.986496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.987023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:51.987707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:51.987988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:51.988266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:51.989024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:51.989331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:51.990719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.004250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 326/641] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:52.004437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.004535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.004638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:52.024690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:52.025616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:52.026464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:52.026464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:52.026464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:52.026464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:52.026464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 316/642] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:52.026848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.125020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.128918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6920102-64b7-4075-9ac3-6debcb36addf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.130342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21686ea-5d85-4f39-b879-37bf6e8a8ee4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.138618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.169928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.171023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.171305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.171579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.174134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.175312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.175592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.175864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.182766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.183583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:52.184274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:52.191478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:52.192441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 317/643] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:52.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:52.199092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:52.199876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:52.200823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 327/644] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:52.201381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.308706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753a461-1326-42bd-882c-4f67b0915939 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.314491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387b1da-1415-4911-b898-a1e815cbedee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.354110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.354910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.355191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.355462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.356196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:52.356555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.357221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.357903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.358056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.358310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.358583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.369986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 328/645] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:52.370841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.373163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.375155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:52.393363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:52.394724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:52.395956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:52.395956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:52.395956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:52.395956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:52.395956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 318/646] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:52.395956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.405526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.418454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.494540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f6356-e95e-40c5-8890-c2027eac09f5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.506547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f47d4-624d-4b90-937d-4c1846723fe0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.537402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.538051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.538331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.538603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.539393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:52.539687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.541066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.549978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 329/647] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:52.550833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.555265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.555938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.556210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.556477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.557271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:52.557573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.559089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.573269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 319/648] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:52.574230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.661613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.665363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.688356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c794d38-c58e-462b-906b-ef6616511065 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.696800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc40b88-861a-479b-8361-2f51965ac95b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.738186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.738940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.739221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.739321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.739502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.740151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.740305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:52.740721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.740955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.741026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.741929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.749985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.751500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.758075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 320/649] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:52.759164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:52.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:52.769370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:52.770292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 330/650] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:52.771965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.877395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a23b13-352d-4713-8338-e36dfd2d5cbb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.885932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23fbfe-ea77-4e9f-9038-b8debd37ab6d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:52.914732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.915565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.915842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.916116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.923830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.924301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92778b81-b092-4319-a083-74d172ded84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:52.925547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778b81-b092-4319-a083-74d172ded84c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:52.925885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778b81-b092-4319-a083-74d172ded84c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:52.926162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92778b81-b092-4319-a083-74d172ded84c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:52.927131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92778b81-b092-4319-a083-74d172ded84c None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:52.927487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92778b81-b092-4319-a083-74d172ded84c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.928985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92778b81-b092-4319-a083-74d172ded84c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.929520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:52.931392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:52.935665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 331/651] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/role_assignments?user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:52.936323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92778b81-b092-4319-a083-74d172ded84c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:52.946745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:52.972698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:52.973669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:52.974510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:52.974510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:52.974510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:52.974510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:52.976394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 321/652] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:52.978065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.061300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92778b81-b092-4319-a083-74d172ded84c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.102635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.103241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.103520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.103800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.113117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.114777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.119656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d685520-0ca9-4b70-8f56-d71e7e716b44 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:53.131023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:53.131967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 14:26:53.132962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:53.137084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 332/653] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:53.138132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.173670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.174849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.175242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.175460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.176576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:53.176889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.179885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.201726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 322/654] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 25 14:26:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:53.202547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.236296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.260925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.282522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ccdc-bde6-4caa-8ea4-b097cda8d735 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.317375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c884f-c3cd-4252-abec-e9cbadc9d2ef None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.327087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.328383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.328823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.329538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.330725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:53.331186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.333246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.349577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 333/655] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:53.350745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.356660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.357353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.357632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.357904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.358787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:53.359105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.360549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.379526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 323/656] 10.222.0.32 () {56 vars in 1137 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:53.380704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.471766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf01a2-d64a-43ba-8ddb-8c91e1d3b149 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.485909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1c8bc-3592-4e20-8541-fe21aeed3133 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.493657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.509780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.511024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.511534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.512027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.515266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.523859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.524674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.524951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.525218 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.533663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.535250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:53.551477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:53.552644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 14:26:53.553554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:53.557781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 324/657] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:53.558863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.600578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.602402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.668492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0223d-fe37-4ab0-a5b6-b58bee5d1098 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.704712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d34bb2-d80d-44a2-a796-918ef8a514af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.705474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34bb2-d80d-44a2-a796-918ef8a514af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.705757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34bb2-d80d-44a2-a796-918ef8a514af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.706029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d34bb2-d80d-44a2-a796-918ef8a514af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.715497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 334/658] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:53.716626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.749500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.755304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.791888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d34bb2-d80d-44a2-a796-918ef8a514af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.794384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d34bb2-d80d-44a2-a796-918ef8a514af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.835299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bd358-6333-4b72-91b6-16f1d2e44ba5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:53.869967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541711b3-f388-4773-99d5-208922155978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:53.870666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541711b3-f388-4773-99d5-208922155978 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.870969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541711b3-f388-4773-99d5-208922155978 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.871237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541711b3-f388-4773-99d5-208922155978 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.872072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541711b3-f388-4773-99d5-208922155978 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:53.872378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541711b3-f388-4773-99d5-208922155978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:53.873809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541711b3-f388-4773-99d5-208922155978 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:53.896600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 335/659] 10.222.0.32 () {56 vars in 1139 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:53.896600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541711b3-f388-4773-99d5-208922155978 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.910160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 325/660] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:53.910160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e14b2-0cad-4411-a0d6-9eca66be8dd9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:53.910160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e14b2-0cad-4411-a0d6-9eca66be8dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:53.910160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e14b2-0cad-4411-a0d6-9eca66be8dd9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:53.910160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 326/661] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:53.910160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e14b2-0cad-4411-a0d6-9eca66be8dd9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:53.981718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:53.989154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.006544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541711b3-f388-4773-99d5-208922155978 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.019177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e14b2-0cad-4411-a0d6-9eca66be8dd9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.041841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.042724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.043031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.043303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.054225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27788b0e-4ced-49d0-8b05-c8227b226813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.054387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.054897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788b0e-4ced-49d0-8b05-c8227b226813 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.055174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788b0e-4ced-49d0-8b05-c8227b226813 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.055439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27788b0e-4ced-49d0-8b05-c8227b226813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.055920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:54.075096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:54.076049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:54.076900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:54.076900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:54.076900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:54.076900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:54.076900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 336/662] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:54.077179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.132886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27788b0e-4ced-49d0-8b05-c8227b226813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.134344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27788b0e-4ced-49d0-8b05-c8227b226813 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.181196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c679019-074a-4414-ae98-942fe49ab617 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.198697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.218772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.219686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.220032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.220320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.223700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.236263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.238098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.243775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 327/663] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:54.243775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.243775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.243775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.259750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:54.260891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:54.261738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:54.262605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:54.263533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:54.263533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:54.263533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:54.263533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:54.263533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 337/664] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:54.264470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.336941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 328/665] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:54.337743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.384893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2aec43-3c8f-4392-b8b7-32274ccbdbd4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.423973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.424988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.425270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.425541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.426275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:54.426667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.428206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.439631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.445450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 338/666] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:54.445450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.450498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43662746-fb3b-45c8-bc2c-c6a32002109c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.493193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.494042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.494532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.494679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.494846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.507330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.508781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:54.526770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:54.527656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:54.528531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:54.528531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:54.528531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:54.528531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:54.532578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 329/667] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:54.533168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.575506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345afb-5e7b-49dd-90a2-f8e9dca35fb8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.632391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.633545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.633995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.634480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.635916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:54.636461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.638036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070245e-a9bf-471f-ac55-8f5cdfec85b1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.639065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.659976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 339/668] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:54.659976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.687539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.688341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.688619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.688899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.699543 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:54.707292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:54.708398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:54.709217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 330/669] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:54.709885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.721791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.816460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780bc63-af12-441e-82bb-c9e95a12fbc0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.828327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:54.849686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02024a6b-58d9-4dc7-941c-4df53ed79e97 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:54.857556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.858363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.858639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.858936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.866586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.868395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:54.890567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:54.891711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:54.892761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:54.898956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:54.899135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 340/670] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:54.900109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.900484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:54.900906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:54.901193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:54.901927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:54.902230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:54.904667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:54.918207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 331/671] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:54.919076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:54.993374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.021663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f0f9-115b-4691-9b0b-5eaa5bb2d149 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.028449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8090a-724c-4d5b-93f9-eeefb5fea027 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.058437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af919496-e180-4099-a222-908b1492cedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.059288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af919496-e180-4099-a222-908b1492cedd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.059662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af919496-e180-4099-a222-908b1492cedd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.059945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af919496-e180-4099-a222-908b1492cedd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.064882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.065559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.065836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.066264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.067180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af919496-e180-4099-a222-908b1492cedd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.067363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.067692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.068623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af919496-e180-4099-a222-908b1492cedd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.069022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.078419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 332/672] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.079629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-af919496-e180-4099-a222-908b1492cedd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:55.085177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:55.086054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:55.086962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 341/673] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:55.087892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af919496-e180-4099-a222-908b1492cedd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.121744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.198414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb4abc-04a6-449c-b068-5b6c8779121e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.208083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af919496-e180-4099-a222-908b1492cedd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.228839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.235694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.236367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.236648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.236924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.237823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.238133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.239597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.246195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.247334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.247604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.247872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.252978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 333/674] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.253655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.259987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.261784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:55.277941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:55.278930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:55.279848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 342/675] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:55.280322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.381722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.387687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0349d50-6135-410d-924d-a4f32fb3dc2c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.393962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a926de-7c03-4b8c-9c77-a37c3d3d2d85 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.424133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.425265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.425544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.425816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.431337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.431995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.432272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.432536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.433324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.433623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.435095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.436852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.438519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.448115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 343/676] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.449061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:55.457611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:55.458541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:55.459461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:55.459461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:55.459461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:55.459461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:55.459461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 334/677] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:55.460010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.512793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.569151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07978f-f9f1-4d53-a824-ec7c167aa6f7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.572388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78fd17-c1d9-4851-a8e9-a2d9b193fa68 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.610632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.611332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.611622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.611843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.611951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.612504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.612727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=cc3c334dc46b47a3b8e415e913c29249) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.612835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.613033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.613123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.613816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.614117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.614487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0525c-c9a1-42df-b250-9a90605f6068 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.615565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.620654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 344/678] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/role_assignments?user.id=cc3c334dc46b47a3b8e415e913c29249&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.621947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25888249-805c-4acb-a410-ddde4fb73c24 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.622231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25888249-805c-4acb-a410-ddde4fb73c24 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.622512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25888249-805c-4acb-a410-ddde4fb73c24 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.623788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 345/679] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:55.624494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25888249-805c-4acb-a410-ddde4fb73c24 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.634185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 335/680] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.636500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.659213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.761704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25888249-805c-4acb-a410-ddde4fb73c24 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.771920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dc729-d2a9-4b85-bcef-2e335947c6f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:55.773643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.806815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.807496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.807817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.808089 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.808951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=175dd1ccd6104b18aea8e69bb5ec036a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.809260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.810771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c654a52-33c4-4b72-b159-9cbf0b978849 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.817187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 346/681] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/role_assignments?user.id=175dd1ccd6104b18aea8e69bb5ec036a&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.819469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.819469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.819469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.821695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:55.822738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:55.823335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:55.823610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:55.824394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:55.824697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:55.826101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:55.841559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 336/682] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:55.842917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.845383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:55.846773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:55.934025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 347/683] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:55.935009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:55.946326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:55.972986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea81f-59e2-40ec-b991-9a5b87142bdf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.015052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.015731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.016011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.016280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.017183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=db335e266cd14d45868e6875a9ea34d4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:56.017440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:56.019146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:56.026271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 337/684] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/role_assignments?user.id=db335e266cd14d45868e6875a9ea34d4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:56.027330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.071815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:56.077124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d89420-5c0a-4ef9-9133-11b649ab2b75 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.118233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.119406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.119858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.120396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.129554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:56.131529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:56.149108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:56.150310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:26:56.151512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:56.153988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 348/685] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:56.161210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.169330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46332-1a96-4eeb-a516-b03210e3804a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.221905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.222784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.223088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.224342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.236404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:26:56.243677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:26:56.246156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:56.247397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:56.248530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 338/686] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:56.254167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.291526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a3460-9e60-419e-a4ac-95ad262fb6cf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.329722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.330431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.330684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.330988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.331903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:56.332145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:56.340006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:56.361201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 349/687] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:56.369239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.371417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:56.389965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fb33f-328c-4f5b-b01f-9fa6d124e8d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.428146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.429074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.429530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.429947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.430873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:56.431340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:56.433009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:56.443540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 339/688] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:56.449960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.490056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1d725-204a-4f42-a38a-2f210ad0ab6f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.516245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:56.528203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.528964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.529313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.529654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.559286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b2bf4-1080-4edb-a9ae-af4552f561ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.594516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.595629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.596129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.596550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.607114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:56.608697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:56.609057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:56.610854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:56.629499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:56.648031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:56.648982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:56.649843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:56.649843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:56.649843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:56.649843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:56.659217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 340/689] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:56.665966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.731431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 350/690] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:56.737630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:56.772372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:56.911569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d875163-1be0-4a84-9c01-ce93d49887a4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:56.981768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:56.982580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:56.982883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:56.983155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:56.999280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.001170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:57.044720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:57.045675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:57.046554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:57.046554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:57.046554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:57.046554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:57.049555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 351/691] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:57.055183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.077193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406375a9-9dd9-4dcb-a309-fbcfdedeb99c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:57.113751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:57.114414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:57.114689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:57.114981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:57.115707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:57.116004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.122802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.137708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 341/692] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:57.144322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.181326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:57.219877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e746a-4cff-4353-8bd7-2beb8507ae24 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:57.274852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:57.275941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:57.276512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:57.276989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:57.277925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:57.278432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.280067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.291125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:57.293541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 352/693] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:57.300184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.364015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab6670-9231-4486-82ac-9b98c7407768 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:57.412122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787d83-e642-42c9-8776-6488caa52043 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:57.428497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:57.429310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:57.429587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:57.429851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:57.442609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.450580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:57.451397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.451922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:57.452429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:57.452954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:57.461451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.464066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:26:57.469884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:57.470874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:26:57.471790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:57.479595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 342/694] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:57.486220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:57.487123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:57.488000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:57.488424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.490668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 353/695] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:57.490938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:57.496850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.630562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfd10c-0612-47ef-8e88-9f8d9a794037 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:57.677544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:57.680685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:57.690617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:57.691611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:57.693518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:57.694460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.697515 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.739236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 354/696] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:57.743833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.763836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf01038-05b3-4ed4-9bd9-bc542d700fc4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:57.821067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:57.831484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:57.833596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:57.833596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:57.833596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:57.834182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:57.834496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:57.835992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:57.872922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 343/697] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:57.878088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:57.899192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:57.979047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75106b77-7cdb-4205-881e-d30fa65eb5b7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.046258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1a39e-0fc3-477e-a4d8-6db539ecdbc2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.048414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:58.049092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.049710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.049978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.050843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=5808b2b191a14c358ec62171be87a01f) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:58.051152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:58.052621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db000d-df09-4e13-9d1b-02f27e77eadd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:58.059781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 355/698] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/role_assignments?user.id=5808b2b191a14c358ec62171be87a01f&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:58.095940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:58.096628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.096905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.097182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.185555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:58.211726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:58.213222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:58.288447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee321f-2827-4cac-978f-a1a359bc5305 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.288729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee321f-2827-4cac-978f-a1a359bc5305 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.289005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee321f-2827-4cac-978f-a1a359bc5305 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.290542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 356/699] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:58.296695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.297192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.297694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.318324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:58.319641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:58.349420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 344/700] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:26:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:26:58.355107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:58.411260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 357/701] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:26:58.445254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:58.467101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:58.490119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39209-afb2-4855-8bb2-98bb48243c90 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.530790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:58.530790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.530790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.530790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.532775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:58.541525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:58.543139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:58.561646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:58.562590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:58.563501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:58.563501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:58.563501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:58.563501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:58.563501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 345/702] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:58.570057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:58.589411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2c07-13af-4fb8-a4d4-e5d82d8db658 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.629034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:58.629882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.630159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.630432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.631127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:58.632557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:58.662560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:58.663616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:26:58.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 358/703] 10.222.0.32 () {56 vars in 1129 bytes} [Wed May 25 14:26:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:58.675756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:58.766949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52afa-59af-46ff-9fef-231b19812529 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.833229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:58.846654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:58.847399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.847679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.847952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.848292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d72ed3-f6d3-41bd-be00-f8df60948081 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.848691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:58.848998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:58.850375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:58.868787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:58.874871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 346/704] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:26:58.880249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:58.883776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:58.884457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.884787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.885060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.885896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:58.886207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:58.887698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed12148-f366-4e48-99a7-30fc85b3d8f2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:58.893968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 359/705] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 25 14:26:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:58.895710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0972d4-a4ce-4c8b-af60-26bccc16d95f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.896253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0972d4-a4ce-4c8b-af60-26bccc16d95f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.896747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0972d4-a4ce-4c8b-af60-26bccc16d95f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.902648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 360/706] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:58.904327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:58.904835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:58.905305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:58.926734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:58.928084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:58.995091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240350e2-aef3-4c83-931b-4dd60e092b68 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:58.998577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 361/707] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:59.006127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.030361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.031113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.031387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.031647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.039342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.041343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-119d3bfd-ca9d-400d-b9f3-f0b1961dd0e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:26:59.058439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:59.059388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 347/708] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc7010-ef6d-4548-9f80-433692cc1354 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.065515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc7010-ef6d-4548-9f80-433692cc1354 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.066174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc7010-ef6d-4548-9f80-433692cc1354 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.066174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 348/709] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:26:59.066174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc7010-ef6d-4548-9f80-433692cc1354 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.118486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e96bbc-bca2-4965-873e-5eae7cc85aae None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.121534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:59.129827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:59.160185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.160866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.161147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.161422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.162142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:59.162451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.163881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.169951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 362/710] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 25 14:26:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:59.170775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.187980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc7010-ef6d-4548-9f80-433692cc1354 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.237476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423564-364e-43aa-956f-c84579f582df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.238958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423564-364e-43aa-956f-c84579f582df None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.239954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423564-364e-43aa-956f-c84579f582df None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.240425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423564-364e-43aa-956f-c84579f582df None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.241623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24423564-364e-43aa-956f-c84579f582df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.244147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24423564-364e-43aa-956f-c84579f582df None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-24423564-364e-43aa-956f-c84579f582df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:59.261844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:59.262974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:59.264039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 349/711] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:59.264980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.265256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.265509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.284263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:26:59.285497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:26:59.304662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd415f-3833-41c4-86cd-d13efa68af02 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.353862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.355681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.356146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.356591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.358243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:59.359019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.360911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.367215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 350/712] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:26:59.368283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.376117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 363/713] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:59.381960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.416938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:59.432479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:59.493879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688c138-9962-4a24-8ac9-6310675b65bd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.519408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f214a2b-f977-4a6b-9742-ffbba348edd7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.529057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.529961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.530340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.530710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.531586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:59.531995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.533539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.539502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 351/714] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:59.540621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.566485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.567197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.567481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.567761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.569397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:59.569723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.571757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.579662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 364/715] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/role_assignments?user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:59.581432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:26:59.709921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:59.715384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf014a9-2b9e-4666-97b3-c997ed69d119 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.759860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.760912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.761363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.761797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.762664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] RBAC: Adding query filter params (type=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:26:59.763188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.764755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.768577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12f5d0-b093-4f1e-82a0-e61e1044c517 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:26:59.771022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 365/716] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:26:59.789457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:26:59.841937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:26:59.843167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:26:59.843637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:26:59.844058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:26:59.859026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:26:59.860779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:26:59.881021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:26:59.882176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:26:59.883307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:26:59.883307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:26:59.883307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:26:59.883307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:26:59.885286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 352/717] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:26:59.891560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:00.058956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399c0d8-6770-4e54-93bd-6162ca9ffc2d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:00.095755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:00.096456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:00.096737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:00.097012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:00.097745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:00.098091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:00.099494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc10907-cf40-481d-8f1f-2bac9dbd8e37 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:00.111790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 366/718] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:00.117262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:00.234124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93a507-dbe9-4179-8adb-02dfadbd9ce4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:00.270611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67502f-77ca-4d37-a492-cc7af21c26ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:00.271638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67502f-77ca-4d37-a492-cc7af21c26ec None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:00.272099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67502f-77ca-4d37-a492-cc7af21c26ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:00.272554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67502f-77ca-4d37-a492-cc7af21c26ec None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:00.273488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67502f-77ca-4d37-a492-cc7af21c26ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:00.275149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67502f-77ca-4d37-a492-cc7af21c26ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:00.300648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 353/719] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:27:00 2022] POST /identity/v3/users => generated 254 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:00.846995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6cc39-ccee-4fdf-a34e-57bd17cba239 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:00.848065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6cc39-ccee-4fdf-a34e-57bd17cba239 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:00.848549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6cc39-ccee-4fdf-a34e-57bd17cba239 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:00.850609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 367/720] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:00.856669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:00.857217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:00.857682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:00.875601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:00.877129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:00.889029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:00.954864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 354/721] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:27:00.978808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6cc39-ccee-4fdf-a34e-57bd17cba239 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:01.001540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:01.093505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6cc39-ccee-4fdf-a34e-57bd17cba239 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:01.138847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:01.139666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:01.139942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:01.140207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:01.159955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:01.168991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:01.169977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:01.170854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:01.174868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 368/722] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:01.177121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:01.306541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c7211-7fad-41f2-8e94-d81675c713ed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:01.357803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:01.358435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:01.358716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:01.359020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:01.359810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:01.360117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:01.362342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:01.375507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 355/723] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:01.381685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:01.512242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:01.539016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99166b-bf4c-42eb-936b-d32c56842e47 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:01.588940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:01.589978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:01.590314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:01.590631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:01.607548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:01.609129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-01fe35ac-4540-4bad-a343-4d0dad1c43a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:01.633602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:01.634602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:01.635724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:01.635724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:01.635724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:01.635724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:01.638877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 369/724] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:01.641567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:01.674925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:01.804047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106826-7121-4a34-bec2-3344fb40ff30 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:01.858269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:01.858998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:01.859338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:01.859557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:01.860307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:01.860613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:01.861998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:01.875281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74091d-0bcb-4467-b2b5-b57955df4ee6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:01.876147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74091d-0bcb-4467-b2b5-b57955df4ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:01.876622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74091d-0bcb-4467-b2b5-b57955df4ee6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:01.878379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 370/725] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:01.881737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 356/726] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:01.884265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:01.884712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:01.885410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:01.886264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:01.921124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:01.922711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:02.000253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 371/727] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 14:27:02.047147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:02.053468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:02.092289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063dba9-0221-41f7-ad76-01bca2db2887 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:02.151193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:02.189013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3b24d-c990-4f5e-a513-0d2654351523 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:02.192881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:02.193703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.193979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.194241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.214901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:02.216840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:02.232982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:02.233990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.234418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.234869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.235779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:02.237824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:02.255479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:02.256760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:27:02.257893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:02.262199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 357/728] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:02.264401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 372/729] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:02 2022] POST /identity/v3/services => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:27:02.272311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:02.405099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4fe19-ba1c-4f2c-b9b6-fb2906fcfd59 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:02.439548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:02.455814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:02.456521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.456849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.457128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.458108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:02.458419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:02.460053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e180793-473f-45cc-b6d1-466950c16c5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:02.481234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 358/730] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 25 14:27:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:02.488223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:02.547499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169148aa-05ac-4dc6-9a5f-09803b3e1e6a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.547836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169148aa-05ac-4dc6-9a5f-09803b3e1e6a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.548057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169148aa-05ac-4dc6-9a5f-09803b3e1e6a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.549788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 359/731] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:02.568779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.569125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.569314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.607672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:02.608939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:02.636870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb6d66-4bda-4132-8a06-3fa100667663 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:02.665039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:02.680172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8739d5d-7feb-4ad9-bfb7-4ff56345beab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:02.682333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8739d5d-7feb-4ad9-bfb7-4ff56345beab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.682791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8739d5d-7feb-4ad9-bfb7-4ff56345beab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.683208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8739d5d-7feb-4ad9-bfb7-4ff56345beab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.703297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 360/732] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:27:02.727878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:02.768198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8739d5d-7feb-4ad9-bfb7-4ff56345beab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:02.770091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8739d5d-7feb-4ad9-bfb7-4ff56345beab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:02.831803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554cbd4-3bcc-4b48-b744-f53d13fc7674 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:02.847760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:02.871320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63ceee-62cc-4419-bc7d-7faf95e6fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:02.872125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ceee-62cc-4419-bc7d-7faf95e6fff5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.872406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ceee-62cc-4419-bc7d-7faf95e6fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.872677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63ceee-62cc-4419-bc7d-7faf95e6fff5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.883446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63ceee-62cc-4419-bc7d-7faf95e6fff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6f63ceee-62cc-4419-bc7d-7faf95e6fff5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:02.891268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:02.892210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:02.893054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 361/733] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:02.894226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfecdb-650d-42a5-81a6-d2a3ac1179d2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.894353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfecdb-650d-42a5-81a6-d2a3ac1179d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.894616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfecdb-650d-42a5-81a6-d2a3ac1179d2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:02.901018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 362/734] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:02.901799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfecdb-650d-42a5-81a6-d2a3ac1179d2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:02.907452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 373/735] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 14:27:02.909443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:02.909933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:02.910498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.027205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfecdb-650d-42a5-81a6-d2a3ac1179d2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.067818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.068793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:03.069629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.070022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:03.070526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.071098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.072239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:03.072753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.074517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.085216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 363/736] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:03.086404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.087938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.127515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.156380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 374/737] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 14:27:03.157261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.200644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61248d84-3eaa-49a3-bbc3-d458c1e6d6db None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.239456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.240794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.241284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.241757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.254796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.256847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.272411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af6895-3944-48dc-8fb4-49d5d6f803ff None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:03.278577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:03.279624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:03.280506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:03.280506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:03.280506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:03.280506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:03.282930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 364/738] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:03.284196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.310650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.312110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.312631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.313203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.326595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.328179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:03.347469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:03.348447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:03.349322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:03.349322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:03.349322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:03.349322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:03.349322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 375/739] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:03.350053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.353890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.385763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.404002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f931d-2ab8-48ea-8cef-88115ff80b7c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.444137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de031814-278a-49ef-adce-315442240831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.445783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de031814-278a-49ef-adce-315442240831 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.446269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de031814-278a-49ef-adce-315442240831 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.446736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de031814-278a-49ef-adce-315442240831 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.447794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de031814-278a-49ef-adce-315442240831 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.449523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de031814-278a-49ef-adce-315442240831 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.460672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9a441-e879-4cf0-967b-f11eca345b2a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.465089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 365/740] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:03 2022] POST /identity/v3/services => generated 240 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:03.465734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de031814-278a-49ef-adce-315442240831 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.514258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.515009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.515290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.515561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.516351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:03.516700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.518261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de00233-4b35-49e1-b086-82bcacc3700f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.533384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 376/741] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:03.534862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4155-c238-4bb1-81a5-394d559715bd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.535164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4155-c238-4bb1-81a5-394d559715bd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.535440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4155-c238-4bb1-81a5-394d559715bd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.537234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 377/742] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:03.545862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.546178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.546420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.575730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:03.577478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:03.619171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de031814-278a-49ef-adce-315442240831 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.658825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.665694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.666580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.667054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.667358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.668080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:03.668383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.669798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.682795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 378/743] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:03.683457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.686187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 366/744] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:03.692628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.703975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.810176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8724246-07d6-4035-9b4d-b0c1096486df None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.824864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3daf231-9086-4a22-ac6f-a3f761483980 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:03.852514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.853439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.853779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.854164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.861721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.863425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.866502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:03.867929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:03.868458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:03.868983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:03.878297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:03.879635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:03.880736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:03.881284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:03.884853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 379/745] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:03.884853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.886661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:03.903794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:03.904792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:03.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 367/746] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:03.906291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:03.954124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:03.987128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:04.010565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91130447-617c-4a23-acdf-3f0316a0d1e7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.033582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9ffe2-2ddf-4014-930f-d8299ff46488 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.046632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.047554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.047893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.048227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.059856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:04.068111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:04.069282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:27:04.070294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:04.074832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 380/747] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:04.076025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.079727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.080860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.081298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.082655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.083650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:04.084100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.085681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.105763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 368/748] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:04.105763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.193559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de9f47-841c-4178-8da5-13d7242b974e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.204626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:04.210503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d8730-9758-4c7f-9d4b-d182a67ce309 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.231827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.232818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.233240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.233669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.234655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:04.235119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.236837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.239720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:04.250433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.251418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.251837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.252248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.253201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:04.253664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.255252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.260431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 381/749] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:04.260431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.274021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 369/750] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:04.274021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.397170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc47d4-6032-4d41-b914-f511c2961613 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.399229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687eb1b-8620-4864-99ed-15e2e1f130b5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.444067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.445065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.445197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.445967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.446202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.446302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.446507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.446939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.448040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=5808b2b191a14c358ec62171be87a01f) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:04.448519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.450131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.462789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.462789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.463677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 382/751] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/role_assignments?user.id=5808b2b191a14c358ec62171be87a01f&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:04.464808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:04.481407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:04.482574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:04.483721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:04.483721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:04.483721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:04.483721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:04.483721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 370/752] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:04.490126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.537454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:04.544824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:04.618733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14a4f-05b8-43dc-9f02-92799b38c0d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.661901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.662960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.663459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.663928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.664866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:04.665381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.667493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.685839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 371/753] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:04.692338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.736925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2816f-476d-4313-afe9-d7f4c9896ccf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.813941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.816781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.817182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.817442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.827028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:04.829752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.831569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.831863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157f96-2589-414a-931b-5d959262cad0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:04.852451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:04.853387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:04.854281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:04.854281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:04.854281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:04.854281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:04.859578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 383/754] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:04.860490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.879985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:04.881194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.881800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.882637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.892429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:04.894668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e658f8d1-14a8-44d6-babe-21625dd2e0c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:04.912687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:04.913566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:04.914422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:04.917646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 372/755] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:04.923549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa33cc-07e0-4f71-aff9-791b9d9d0b91 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:04.923549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa33cc-07e0-4f71-aff9-791b9d9d0b91 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:04.923549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa33cc-07e0-4f71-aff9-791b9d9d0b91 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:04.925285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 373/756] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:04.925869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa33cc-07e0-4f71-aff9-791b9d9d0b91 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:04.982054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.003334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc92d0-d0e6-4e13-8ff3-785a2cb69032 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.040716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.041802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.042424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.043119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.043592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa33cc-07e0-4f71-aff9-791b9d9d0b91 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.044280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:05.044777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.046386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550cb96-73ab-4dfd-89c3-79a81732682a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.059879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 384/757] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:05.061476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.061950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.062413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.079927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:05.080548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.081268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.081547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.081822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.082323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:05.082661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:05.083018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.084503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.097147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 374/758] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:05.102349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.129094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.169902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 385/759] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:05.171065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.213378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cd9f1-c457-418b-9a04-a7d2b7e93948 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.252208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.254816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.256245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.256949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.257392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.268701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.270917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.279224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7bc9d-0b75-4e4c-9bb6-15858fe24fe7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:05.291283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:05.292469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:05.293584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:05.296428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 375/760] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:05.297420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.324038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.325124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.325528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.325889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/eeee4e4e0fc74d7db8b5cb9d0ba806a4/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.328083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=eeee4e4e0fc74d7db8b5cb9d0ba806a4, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.329683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.354795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 386/761] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:05 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/eeee4e4e0fc74d7db8b5cb9d0ba806a4/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:27:05.354795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.412257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.439984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9181767-4174-4885-97db-997062aa26b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.485663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.486499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.486807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.487117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.487785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.489191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:05.517897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:05.518889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:27:05.519706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:05.521064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 376/762] 10.222.0.32 () {56 vars in 1129 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:05.528098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.609702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd894112-a695-4601-8ad8-f175c802aa28 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.646509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b63464-6bfb-4a33-b585-a44292434b1a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.654527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.655605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.656131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.656579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.658451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:05.658992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.660687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.663108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.682982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 387/763] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:05.691251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.691789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.692491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.692770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.693166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.693902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:05.694203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.695666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.701924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 377/764] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:05.707774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.751181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.815008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c03df-77fe-435c-a27a-03b1943dd477 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.822836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e771ee-4e03-499d-b197-d0186a6fa284 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:05.861465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.862104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.862381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.862650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.867788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:05.868105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.869685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.883683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46dc79-a4fe-48dd-91e7-14fcb4cf002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:05.884987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46dc79-a4fe-48dd-91e7-14fcb4cf002e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:05.887040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46dc79-a4fe-48dd-91e7-14fcb4cf002e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:05.889344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 378/765] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:05.889992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46dc79-a4fe-48dd-91e7-14fcb4cf002e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:05.896143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:05.975880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:05.987098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46dc79-a4fe-48dd-91e7-14fcb4cf002e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:05.988776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46dc79-a4fe-48dd-91e7-14fcb4cf002e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:05.990288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:06.024698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959b0e-0ca7-4d84-aaf4-9dc30554e005 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.069150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.069837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.070244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.070517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.071362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] RBAC: Adding query filter params (interface=public, service_id=de23fdf62b6d4a8c9bbec27dd92b8f3d, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:06.071666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.073093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49691df7-44af-4d85-ae15-4bceba3f390a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.081910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 379/766] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/endpoints?service_id=de23fdf62b6d4a8c9bbec27dd92b8f3d&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:06.083427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8e725-14fc-48b7-b8a2-78c79fa03e19 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.083709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8e725-14fc-48b7-b8a2-78c79fa03e19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.083979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8e725-14fc-48b7-b8a2-78c79fa03e19 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.085760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 380/767] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:06.086557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8e725-14fc-48b7-b8a2-78c79fa03e19 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.111409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 388/768] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:06.112756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.113044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.113465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.131510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:06.132706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:06.204526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8e725-14fc-48b7-b8a2-78c79fa03e19 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.215162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 389/769] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:06.216108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.255316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.255980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:06.256718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.257107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.257383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.258461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.260931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.263954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:06.269606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 381/770] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:06.271421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.394522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bea464-7d96-461a-9490-da16b0fdc243 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.426036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e9c25-b433-424a-9bf2-ff81722eafbd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.437901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.438736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.439088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.439375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.440036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.441421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:06.456420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:06.460191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:06.463542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 382/771] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:06.475626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.537956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.538675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.538996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.539237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.543864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:06.571750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.573325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.588444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba990e-bb49-430d-90a9-9532122ac7be None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:06.610114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:06.613032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:06.615175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:06.615175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:06.615175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:06.615175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:06.615175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 390/772] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:06.619695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.627566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.628256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.628602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.628930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.629903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:06.630211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.631621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.637985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 383/773] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:06.643522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.712510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:06.744871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f8b41-b442-4446-993b-5bdcda51c97b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.768399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a81a8-c92b-4257-aaa5-2b193d7c29ec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:06.799579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:06.812145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.812829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.813110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.813386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.816866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:06.817188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.818595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.830281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 384/774] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:06.835862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:06.836603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:06.836891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:06.837165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:06.837927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:06.838237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:06.839825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:06.851534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.859898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 391/775] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:06.867479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:06.985200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb37ba-5b3c-469b-9d3b-54ca6b77a82a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.026267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.028535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.028815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.029086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.033069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:07.043716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:07.045203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:07.070860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:07.071787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:07.072736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:07.074357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 392/776] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:07.080551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:07.105728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e54d-7057-4314-9334-6025807492e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.165223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.165907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.166238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.166478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.167319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] RBAC: Adding query filter params (interface=public, service_id=9fba3abdedd44bfe849770c3b91af877, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:07.167633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:07.169018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:07.180256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 385/777] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:06 2022] GET /identity/v3/endpoints?service_id=9fba3abdedd44bfe849770c3b91af877&interface=public®ion_id=RegionOne => generated 194 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:07.186018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:07.272997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aa5f9-4dfa-432d-9089-2ad831294c91 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.304686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:07.342407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.343501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.343976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.344514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.345520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:07.346020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:07.347854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d637a0-0d2b-417a-984a-5e6968e0c10a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:07.361866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 393/778] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:07.363791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6cb0a-02d2-4e5c-96bc-8843803d25e1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.364345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6cb0a-02d2-4e5c-96bc-8843803d25e1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.364814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6cb0a-02d2-4e5c-96bc-8843803d25e1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.366818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 394/779] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:07.367740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6cb0a-02d2-4e5c-96bc-8843803d25e1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:07.371160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db03dc0-9f8c-4028-8168-39f66f0b0235 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.408991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907507f-d05f-4c10-bcea-4994d6908260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.410308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907507f-d05f-4c10-bcea-4994d6908260 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.410308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907507f-d05f-4c10-bcea-4994d6908260 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.410482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907507f-d05f-4c10-bcea-4994d6908260 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.411132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907507f-d05f-4c10-bcea-4994d6908260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:07.412503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907507f-d05f-4c10-bcea-4994d6908260 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:07.418563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 386/780] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:07 2022] GET /identity/v3/services => generated 772 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:07.420135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.420338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.420622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.435793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:07.442136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:07.443487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:07.489396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6cb0a-02d2-4e5c-96bc-8843803d25e1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.520785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 387/781] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:07.527678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.528884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.529362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.529851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.530986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=d61f6d6395f54f32adf1ba1ccdb6bc43) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:07.531506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:07.533160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:07.540067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 395/782] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:07 2022] GET /identity/v3/role_assignments?user.id=d61f6d6395f54f32adf1ba1ccdb6bc43&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:07.544643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:07.604629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:07.666669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa5179-e30c-4210-852b-7129f387864a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.705858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.706691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.707011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.707287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.720028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:07.728095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:07.730410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:07.732347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 388/783] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:07.738805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:07.885046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9e7ef-f521-433d-adf8-1b664e48fccf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:07.917429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:07.951397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:07.952559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:07.953038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:07.953512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:07.954821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:07.955359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:07.957935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d786f7-4a7e-471d-b22e-2c9c78023427 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:07.974421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 396/784] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:07.979935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:08.123818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633dc60-d153-4bee-b5f9-f2341bde699b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:08.124206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618052-bafc-47a7-afad-438f82396b20 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.124689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618052-bafc-47a7-afad-438f82396b20 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.125176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618052-bafc-47a7-afad-438f82396b20 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.128035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 397/785] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:08.136657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:08.137114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.137899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.138069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.160474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:08.161287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.161568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.161845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.174939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:08.176467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:08.194508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:08.195602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:08.196699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:08.196699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:08.196699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:08.196699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:08.196699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 389/786] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:08.201659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:08.208413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:08.210308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:08.308674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51e182-583d-4a7f-939d-6e66eaf4a675 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:08.346631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:08.347520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.347805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.348081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.348848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:08.349157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:08.350585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:08.365340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 390/787] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:08.370981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:08.423245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:08.477745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f56fd-e415-4ea7-b9f2-39d1b6cbc4cc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:08.513984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:08.514819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.515094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.515359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.523583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:08.525288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:08.540297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:08.541410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:08.542482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 391/788] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:08.547588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:08.552645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 398/789] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:08.586703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:08.663236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:08.679598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728290f-7b6d-45bc-a85c-3de35ef47e77 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:08.715902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:08.717098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.717389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.717656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.718505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:08.718898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:08.720754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:08.739824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 392/790] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 25 14:27:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:08.745821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:08.870766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc2d42-73c0-4e96-935a-62f34d96da8c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:08.890945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:08.905618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8a3eb-7e30-4966-9859-4b36b9994dd3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:08.930124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:08.931591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.932232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.932843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.951687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:08.967380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:08.968505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:08.969444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:08.971843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:08.972712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:08.972992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:08.973255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:08.973404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 399/791] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:08.981212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.003166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.014602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.017406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:09.077709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:09.078688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:09.079611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:09.079611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:09.079611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:09.079611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:09.083413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 393/792] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:09.090923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.208952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a91a5b-f1aa-41cd-a346-f454b496d1da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.224437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44beb5ac-7973-44ea-b793-af064b6f42c4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.245664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.246331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.246609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.246922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.247750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:09.248050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.249878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52adc2-f078-4c83-a3f6-c6b41406007e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.262340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 394/793] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:09.264949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.266040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.266542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.267062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.268208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:09.268455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2d49e-4114-4cb1-b8b1-4708d2404897 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.269202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.270101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2d49e-4114-4cb1-b8b1-4708d2404897 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.270386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2d49e-4114-4cb1-b8b1-4708d2404897 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.271623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b515ce-ff6e-4a8d-8c9b-91bd4bfe516e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.275898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 395/794] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:09.275898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2d49e-4114-4cb1-b8b1-4708d2404897 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.287412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 400/795] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:09.289240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.289732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.290205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.308704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:09.310354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:09.335669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.341626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.388833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2d49e-4114-4cb1-b8b1-4708d2404897 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.394891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 401/796] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:09.396210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.449349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.450169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.450449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.450720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.458532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.460070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:09.475895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:09.476827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:09.477688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:09.481952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 396/797] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:09.483383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.517555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab62828-d6c2-4f0a-ac6f-ff94897653dd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.554524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.555738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.556233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.556683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.568262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.570071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:09.589147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:09.590083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:09.591062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:09.591062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:09.591062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:09.591062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:09.591280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e67845-33e7-49e5-98a4-e66b740542a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.593753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 402/798] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:09.594826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.603947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.612544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.629356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.630219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.630496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.630785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.631756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.633795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:09.649599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:09.650545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 14:27:09.651539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:09.654853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 397/799] 10.222.0.32 () {56 vars in 1129 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:09.655998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.712011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e22fa-c38b-4f48-a10c-92ff153a5b20 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.755577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.756353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.756635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.756906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.758109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:09.758418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.760240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.778900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 403/800] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:09.779993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.786971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811f14b-711c-41e5-9a08-c2ca3f6235f6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.836036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.837499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.838279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.838797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.839964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:09.840959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.842803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.859999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 398/801] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:09.861983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:09.873114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.907008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70753123-655c-4ea7-959c-e2f4f7d8f26e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:09.915842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:09.960975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:09.961703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:09.961938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:09.962263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:09.962968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:09.963393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:09.964758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:09.976610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 404/802] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:09.976610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:10.013830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110b15e-7aab-4da2-bbd3-7a5a81a6e0f0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:10.060710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:10.062260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.062800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.063268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/5808b2b191a14c358ec62171be87a01f/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.067067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=5808b2b191a14c358ec62171be87a01f, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:10.069142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:10.103518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 399/803] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:09 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/5808b2b191a14c358ec62171be87a01f/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:27:10.103518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:10.156669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3ac35-4917-4320-913d-c47c93c5db5f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:10.241008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:10.241830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.242018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.242288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.267765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:10.269302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:10.279265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:10.295578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:10.300758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f9cde-14d4-4ca2-be96-38bf36cbf6fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:10.311122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:10.312122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:10.313025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:10.319344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 405/804] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:27:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:10.344931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:10.360987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:10.362685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.363236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.364133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.365395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:10.365905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:10.368559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:10.376095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 400/805] 10.222.0.32 () {56 vars in 1124 bytes} [Wed May 25 14:27:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:10.382497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:10.484898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaee89e-13c4-45f9-8796-bd6992fcc391 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:10.497820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dea069-0b77-40aa-8b60-4897895cfec9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:10.535324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7aaf2-5bee-4c99-aaa1-93f0f5786608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:10.536368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7aaf2-5bee-4c99-aaa1-93f0f5786608 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.536811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7aaf2-5bee-4c99-aaa1-93f0f5786608 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.537248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7aaf2-5bee-4c99-aaa1-93f0f5786608 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.538188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7aaf2-5bee-4c99-aaa1-93f0f5786608 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:10.539837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7aaf2-5bee-4c99-aaa1-93f0f5786608 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:10.568037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:10.570936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.571485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.572192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 401/806] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:10.572746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.573908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:10.574396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:10.576079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:10.578501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae91702-16bc-41ea-99ba-1833ecad904c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.579102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae91702-16bc-41ea-99ba-1833ecad904c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.579572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae91702-16bc-41ea-99ba-1833ecad904c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.581296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 402/807] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:10.587812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5e41d-da62-42eb-9f61-bdbfb7490bf4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.588472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5e41d-da62-42eb-9f61-bdbfb7490bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.588927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c5e41d-da62-42eb-9f61-bdbfb7490bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.622500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 406/808] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:27:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:10.623872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93c5e41d-da62-42eb-9f61-bdbfb7490bf4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:10.625712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93c5e41d-da62-42eb-9f61-bdbfb7490bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:10.627962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:10.644224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:10.755034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 403/809] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:10.755272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:10.762949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b98c39-77da-4aa4-8e07-7c4f7266196b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.763333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b98c39-77da-4aa4-8e07-7c4f7266196b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.763573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b98c39-77da-4aa4-8e07-7c4f7266196b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.765221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 404/810] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:10.772091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.772540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.772985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.794998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:10.796219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:10.824276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc34593-de18-456d-b8e8-649ac8fbc4c5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:10.870773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:10.871697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:10.872115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:10.872523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:10.884334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 405/811] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:10.885467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:10.980945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:10.983279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:10.996475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.029539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666af9fc-e9c1-4524-ae6a-644e41ca958e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.045577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.066934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.067748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.068025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.068296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.068954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.070397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:11.082820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:11.084805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 14:27:11.086401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:11.089542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 406/812] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:10 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:11.094818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.095138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 407/813] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:27:11.100429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.194425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b48e72-2960-4c8b-9ab9-5f267ccaf6be None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.208200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a4514-f788-4776-84b3-a7b8f4cde591 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.229703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.230511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.230830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.231101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.234047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.244383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:11.248581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.249861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.250314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.250731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.251622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:11.252121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:11.252299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:11.253376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:11.254268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 407/814] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:11.254991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.256814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.263173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 408/815] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:11.264303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.298403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.390365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176353a-0891-4135-946e-120adfff5a4c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.406342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90c0aa-a7ec-4106-a2f1-7822d3f7d5d7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.432235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.433199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.433576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.434288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.446871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.448651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.451034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.451740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.452020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.452291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.453057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:11.453361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.454772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.469123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 409/816] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:11.470247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:11.477038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:11.478007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:11.479562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:11.479562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:11.479562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:11.479562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:11.479562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 408/817] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:11.479562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.544041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.578718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.602404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cfbd1-eac7-40bc-b8c2-d998e8c3c96f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.616735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fa9a3-86d7-44d1-839e-1785611165cb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.646363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.647650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.648115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.648592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.649670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] RBAC: Adding query filter params (type=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:11.650166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.652536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.659735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 410/818] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:11.660960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.672856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.674114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.674463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.674870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.688014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.689677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:11.720463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:11.721426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:11.722290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:11.722290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:11.722290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:11.722290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:11.725134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 409/819] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:11.726523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.806780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1f298-02f0-4f9c-8ab9-39e0cb665214 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.838724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.839543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.839916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.840282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.841248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:11.841737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.843297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.857705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 411/820] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:11.858792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:11.860087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.863779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3c37e-e447-4f5a-a507-8c3b06b26742 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:11.872375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:11.911053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:11.913187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:11.913822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:11.914465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:11.916629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:11.919701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:11.957390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 410/821] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:11.958734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.053133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662373f-7742-4c2b-acce-691ba8940138 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.119732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.121115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.121604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.122091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.123077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:12.123588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.125707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.134783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6cd1-3080-4fd0-965e-3ac56695f0a1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.162789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 412/822] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:12.166698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.170291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.170291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.170291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.170291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.185392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.199152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:12.225926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:12.226965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:12.227861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:12.231260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 411/823] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:12.240608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.279231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:12.303303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:12.360706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0310e4d-09d5-4db6-8549-fc59bb2eb7b6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.394892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14aa93-7bbb-45db-9eaf-2ec37ad117c3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.408700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.409893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.410622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.411110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.422170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.425428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.436291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.436961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.437239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.437510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.438304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:12.438609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.440072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:12.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:12.447022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:12.447970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:12.451875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 413/824] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:27:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:12.460059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 412/825] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:12.460269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.461294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.546036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:12.582152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ab5ad-d52c-464e-80bc-3e9959eb5c4f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.600984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8eda8-b169-4ea7-81ae-022c923c38d6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.633056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:12.642081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.642932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.643186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.643461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.644396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:12.644708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.646255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.650600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.651631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.652049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.652454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.653391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 413/826] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:12 2022] GET /identity/v3/role_assignments?user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:12.653668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:12.654105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.655754 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.677953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 414/827] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 25 14:27:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:12.686817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.819933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b342eb-8d7a-4130-99f7-83c16acc5c4c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:12.859844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:12.860720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.860999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.861268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.872199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:12.875217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:12.876957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-db25cf37-34c8-4e12-a0f1-b2cead42605c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:12.896225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:12.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:12.898295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:12.898295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:12.898295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:12.898295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:12.898295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 414/828] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:12.903299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:12.981110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de58ae-698d-401d-9ae4-c7ef38d8e003 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.981757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de58ae-698d-401d-9ae4-c7ef38d8e003 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.982263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de58ae-698d-401d-9ae4-c7ef38d8e003 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:12.984161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 415/829] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:12.989961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:12.990244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:12.990516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.009065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:13.010306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:13.019370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46065b0f-7de2-434d-8f11-e72ee62a1387 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.061830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.062055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.062319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.062579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.063381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:13.063690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.065148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.079973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 415/830] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:13.086035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.118170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 416/831] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:13.135771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.141448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:13.204189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985bb2c-10f7-41cd-b2f4-5e62e8b8bfab None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.239835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.240956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.241411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.241854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.242052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3431-0315-4c99-8e95-5f890118866b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.249713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.251504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f2272f76-3c0b-4d3b-a81d-8c27ea9b669a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:13.267063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:13.268081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:13.268969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:13.269391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:13.270082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 416/832] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:13.271715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5cf9-e1ff-4365-a65a-dbea5c26a877 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.272193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5cf9-e1ff-4365-a65a-dbea5c26a877 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.272627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5cf9-e1ff-4365-a65a-dbea5c26a877 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.274364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 417/833] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:13.275279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5cf9-e1ff-4365-a65a-dbea5c26a877 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.281192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a6347-f1f2-4d24-9190-ff2b5e7cadcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.282113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a6347-f1f2-4d24-9190-ff2b5e7cadcc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.282391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a6347-f1f2-4d24-9190-ff2b5e7cadcc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.282656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a6347-f1f2-4d24-9190-ff2b5e7cadcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.293642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a6347-f1f2-4d24-9190-ff2b5e7cadcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a69a6347-f1f2-4d24-9190-ff2b5e7cadcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:13.302403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:13.303432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:13.304399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:13.306590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 417/834] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:13.308186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.308469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.308741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.327554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:13.329110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:13.383005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:13.387538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5cf9-e1ff-4365-a65a-dbea5c26a877 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.404612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 418/835] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:13.405754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.445660 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.447108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.447671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.448106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.449453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:13.449952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.452777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.474109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 418/836] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:13.475422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.519840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b39228-4f7b-44a1-986a-5531a7531e1f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.534453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:13.563074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.563742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.564174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.564445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.565583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:13.566151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.567886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.584542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 419/837] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:13.584542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.589785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07112e1-322a-4070-8076-ef566b32bebb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.629167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.629961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.630250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.630515 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.631197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.632571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:13.644863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:13.645890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:13.647622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 419/838] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:13.651888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.678691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:13.697252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f96df-d74b-4c8c-812d-faa3408f7be7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.732417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.733233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.733514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.733785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/d61f6d6395f54f32adf1ba1ccdb6bc43/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.748666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=d61f6d6395f54f32adf1ba1ccdb6bc43, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.750123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.759553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ee99b-e506-45d9-80d6-1e8718c6da69 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:13.774793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 420/839] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:13 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/d61f6d6395f54f32adf1ba1ccdb6bc43/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:27:13.774793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.805062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:13.805897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:13.806180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:13.806456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:13.820633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:13.825681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:13.827320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:13.848258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:13.849256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:13.850124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:13.850124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:13.850124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:13.850124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:13.850124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 420/840] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:13.856547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:13.965586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080bc2-7a35-41bc-bd0d-9a2a85e57e14 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:14.006916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:14.017757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:14.018324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:14.018608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:14.019726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:14.020453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:14.020763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:14.022160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:14.027748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 421/841] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:14.033488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:14.045765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11753c7-cc65-4a67-b0e4-8ad72ddc88b8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:14.091855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:14.092536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:14.092817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:14.093093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:14.093830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:14.094139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:14.095558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:14.111338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 421/842] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:14.117473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:14.154177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:14.167154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a6655-c5a7-4db4-b7e7-03ef1b72d73b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:14.203686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:14.204365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:14.204644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:14.204917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:14.205637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:14.205938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:14.207353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:14.212905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 422/843] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:14.239874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa966767-6548-4e3f-8d2a-7cbd0d1a5943 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:14.283648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:14.284408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:14.284689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:14.284962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:14.288726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:14.295763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:14.297387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:14.315131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:14.316357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:14.318294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 422/844] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:14.323266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:14.445295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ec7cb-4dda-48f9-a6ce-09c5cb0da8da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:14.483429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:14.484114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:14.484389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:14.484663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:14.485516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:14.485826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:14.487288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:14.507272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 423/845] 10.222.0.32 () {56 vars in 1135 bytes} [Wed May 25 14:27:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:14.513056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:14.625505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d48e16-24e3-41ff-8266-02e5020cabc7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:14.661043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:14.661697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:14.661975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:14.662247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:14.728452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:14.735096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:14.736625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:14.835806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 423/846] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:14.841582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:14.859351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:14.964422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5710dc2-d649-46a1-be78-4cca1f8cc728 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.003462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.004316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.004603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.004901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.017690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.019269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-25846277-4461-4770-88cf-2cd109d32596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:15.038539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:15.040076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:15.042034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:15.042034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:15.042034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:15.042034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:15.042034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 424/847] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:15.051793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.122193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd169f0-6d7c-4108-850b-6da3f940198b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.122560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd169f0-6d7c-4108-850b-6da3f940198b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.122799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd169f0-6d7c-4108-850b-6da3f940198b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.124332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 425/848] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:15.129762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.130067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.130288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.147311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:15.148600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:15.172611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:15.179834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85114c10-2256-485b-a596-981ac5ddb1ad None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.219432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.220480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.220966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.221242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:15.222058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.223126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:15.225015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.225563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.238963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 426/849] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:15.243894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 424/850] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:15.245141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.260689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.363634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8556af5-4d66-4fe8-ac27-cc141e8dd92a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.403613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64208123-9762-4e4a-ac1b-7d9f4c2bfb2c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.408912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.410150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.410635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.411149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.419552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.421354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:15.437689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:15.438635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:15.439541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:15.445276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 427/851] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:15.446407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.448715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.449548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.449836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.450074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.450729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.452143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:15.471756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:15.472854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:15.473699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:15.476782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 425/852] 10.222.0.32 () {56 vars in 1132 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:15.483511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.522173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:15.543860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:15.584776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2466ebc-0861-443d-958c-51b1b9dce393 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.612701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9c49d-43d0-4ade-8ec3-b0f245b21b77 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.628562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.629636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.630135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.630650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.631813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:15.632425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.634074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.647839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 428/853] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:15.650601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.651353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.651638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.651911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.652844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:15.653021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.654116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.654408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.664404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 426/854] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:15.667366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:15.782930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95e3a9-25ba-49f1-bd84-a2f739d32195 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.789598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:15.803464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:15.817376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd968cb9-fd7f-4469-9770-8344d9924f3f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:15.832244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.832915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.833195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.833467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.834317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=5808b2b191a14c358ec62171be87a01f) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:15.834628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.836104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05b63-5a1f-4878-bab4-486b44c1e345 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.843226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 429/855] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/role_assignments?user.id=5808b2b191a14c358ec62171be87a01f&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:15.868744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:15.869599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:15.869812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:15.870062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:15.871027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:15.871331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:15.872732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7b086-1359-4036-a010-0800992a61b9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:15.882938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 427/856] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:16.241351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94bea-827f-4991-9f9d-283b80193d8a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.241771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94bea-827f-4991-9f9d-283b80193d8a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.241944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d94bea-827f-4991-9f9d-283b80193d8a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.243562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 430/857] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:16.250394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de61b-c682-4993-bd2a-badd43d36b28 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.250736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de61b-c682-4993-bd2a-badd43d36b28 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.250996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2de61b-c682-4993-bd2a-badd43d36b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.269712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1f2de61b-c682-4993-bd2a-badd43d36b28 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:16.270973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1f2de61b-c682-4993-bd2a-badd43d36b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:16.368172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 428/858] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:16.392806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94bea-827f-4991-9f9d-283b80193d8a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:16.528107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d94bea-827f-4991-9f9d-283b80193d8a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:16.540160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194252a2-8e9f-4211-975d-8872bbdb8cb6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.540806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194252a2-8e9f-4211-975d-8872bbdb8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.541223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194252a2-8e9f-4211-975d-8872bbdb8cb6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.543046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 429/859] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:16.549844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.550478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.550478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.568973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:16.570598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:16.571747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:16.572807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.573329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.573751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.589025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:16.597890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:16.599084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:27:16.600876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:16.605566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 431/860] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:27:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:16.611061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:16.654547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 430/861] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:16.655501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:16.678616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:16.689324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:16.728259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b347-18b1-4975-b0df-1b2483fe54f9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:16.773815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:16.774679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.775334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.775650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.776420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:16.776732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:16.778177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:16.791141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 432/862] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:27:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:16.797529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f46fb-ff29-4fbe-a876-c86f716c80c2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:16.799033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:16.837010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:16.837723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.838203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.838488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.839278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:16.840938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:16.860093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 431/863] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:16 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:27:16.917634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:16.922097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade97ac-e719-4102-be7b-c6cdd9b87181 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:16.968838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:16.969683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:16.969968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:16.970243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:16.982154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:16.983682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:17.003427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:17.004387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:17.005374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:17.005374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:17.005374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:17.005374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:17.005374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 433/864] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:17.012076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:17.091145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:17.131735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44382bd-5e3a-41cf-9256-798bd841f260 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:17.172267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:17.173895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:17.174407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:17.174902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:17.175890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:17.176421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:17.177996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:17.191619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 432/865] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:17.199683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:17.307504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed1b9b-ea20-4196-923d-1cc3160020a6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:17.348390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:17.349135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:17.349415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:17.349688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:17.357663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:17.359156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:17.381547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:17.382624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:17.384467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 434/866] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:27:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:17.389237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:17.505669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2422db-95f9-456b-81b3-e0910c6e6dd8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:17.536354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:17.548637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:17.549344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:17.549629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:17.549905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:17.550787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:17.551089 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:17.552547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:17.572744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 433/867] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 25 14:27:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:17.577975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:17.587136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:17.679289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea62ac6-60ab-4832-9b50-d5ce4ba4d6a7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:17.713822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:17.714478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:17.714773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:17.715108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:17.791423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:17.793331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:17.803198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:17.898786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 435/868] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:17 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:17.908150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.011568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785add4b-228c-4aaa-b715-5523d388350b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.060204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.061032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.061304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.061569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.073910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.075428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4fbb5490-85a5-4f25-ba78-120649858e96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:18.094054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:18.096074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:18.097680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:18.097680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:18.097680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:18.097680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:18.097680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 434/869] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:18.101077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.216515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed9544-6a76-454b-8689-f39efbb46766 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.227795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:18.244288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537b994-04cd-4b07-b48c-2da67b1f2480 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.244656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537b994-04cd-4b07-b48c-2da67b1f2480 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.244823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537b994-04cd-4b07-b48c-2da67b1f2480 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.247212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 435/870] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:18.257135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.257457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.257993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.262822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.263848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.264328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.264799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.265565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:18.265870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.268174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.278880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:18.280986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:18.286671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 436/871] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:18.295811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.319795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:18.376795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 436/872] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:18.396316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.437275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42af4-bfcf-4817-ae71-31290861690d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.473118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.474179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.474452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.474704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.481914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.483370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ac54009b-30d6-457e-b5d1-88b14512a998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:18.497372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:18.498306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:18.499257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 437/873] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:18.504203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e4c3-eb3a-4e73-b027-ef9b5a7f7344 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.504486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e4c3-eb3a-4e73-b027-ef9b5a7f7344 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.505919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9e4c3-eb3a-4e73-b027-ef9b5a7f7344 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.507384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 438/874] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:18.512031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e4c3-eb3a-4e73-b027-ef9b5a7f7344 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.523757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50546cb7-7774-47ed-b67e-56eef1711285 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.540821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:18.559999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.561133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.561554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.561962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.562625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.564589 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f890388e-6f2e-4d58-a36e-7f608ca8e018 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:18.580547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:18.581613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:18.582443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 437/875] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:18.583780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.583780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.584019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.604533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:18.606006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:18.624662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9e4c3-eb3a-4e73-b027-ef9b5a7f7344 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.634231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:18.661099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.661771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.662050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.662319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.663141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:18.663447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.664884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.677585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 439/876] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:18.678395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.686422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 438/877] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:18.687593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.771420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:18.790207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8c6e-5063-4522-9f36-16b40ae35212 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.804522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56d5ef-1d2d-436a-a521-2951a9d1dfa3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.830957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.831638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.831914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.832216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.832930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:18.833236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.834691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.839956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 440/878] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:18.840738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.842559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:18.843584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:18.844831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:18.845119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:18.845974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=d61f6d6395f54f32adf1ba1ccdb6bc43) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:18.846290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:18.847921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:18.862782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 439/879] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/role_assignments?user.id=d61f6d6395f54f32adf1ba1ccdb6bc43&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:18.862782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:18.898341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:18.971425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d11ed6-fed0-4e7f-8c8e-14abe3b09a97 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:18.997211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da739333-0397-4ad8-82cd-ef40769c1ba9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:19.010772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a460cd-0b2c-4002-af76-cecbe81576ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:19.011550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a460cd-0b2c-4002-af76-cecbe81576ee None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.011899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a460cd-0b2c-4002-af76-cecbe81576ee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.012352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a460cd-0b2c-4002-af76-cecbe81576ee None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.013162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a460cd-0b2c-4002-af76-cecbe81576ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:19.014674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a460cd-0b2c-4002-af76-cecbe81576ee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:19.036713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 441/880] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:18 2022] POST /identity/v3/services => generated 269 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:19.054540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:19.055263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.055549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.055826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.056734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:19.057051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:19.058701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e604c54-b6b2-4167-afc4-6d13d488afe8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:19.065469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 440/881] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:19.241070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f372-bd73-4683-90fd-f263b6d6cfb8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.241605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f372-bd73-4683-90fd-f263b6d6cfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.241605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f372-bd73-4683-90fd-f263b6d6cfb8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.243346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 442/882] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:19.249477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.249764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.250034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.269976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:19.270331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:19.343954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 441/883] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:19.365775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0f372-bd73-4683-90fd-f263b6d6cfb8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:19.379728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:19.473661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0f372-bd73-4683-90fd-f263b6d6cfb8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:19.508656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:19.509468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.509859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.510132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.510882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:19.512236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:19.524902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:19.525965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:19.527016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 443/884] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:19.534399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:19.555756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:19.654115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463dcb8-57d0-4ed8-b71c-9b93bde01a88 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:19.695235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:19.695915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.696194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.696466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.697254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:19.697557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:19.698958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:19.704362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 442/885] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:19.710207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:19.797752 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:19.825670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a880-7eb2-4d5a-8986-f6a5d4f65592 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:19.861442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:19.862118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:19.862393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:19.862664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:19.863470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:19.863772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:19.865152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:19.875071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 444/886] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:19.878562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:19.967212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:19.985538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43a5b5-05be-40e0-a27c-8d22ca10ae9a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:20.020367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:20.021038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:20.021314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:20.021586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:20.022381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] RBAC: Adding query filter params (interface=public, service_id=f6d2994c7a1646b0a0ee878397514697, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:20.022688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:20.024080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2bf60-bcc3-4ec5-8ae7-6441cc2be65b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:20.036043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 443/887] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:19 2022] GET /identity/v3/endpoints?service_id=f6d2994c7a1646b0a0ee878397514697&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:20.037421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:20.152022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb3121-61a0-48f4-b889-63f54b99578b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:20.189222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763b54b-4f14-4284-b3e6-294b03396016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:20.189898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763b54b-4f14-4284-b3e6-294b03396016 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:20.190170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763b54b-4f14-4284-b3e6-294b03396016 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:20.190437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763b54b-4f14-4284-b3e6-294b03396016 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:20.191157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763b54b-4f14-4284-b3e6-294b03396016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:20.192509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763b54b-4f14-4284-b3e6-294b03396016 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:20.202880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 445/888] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:20 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:27:21.255267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb7d29-0944-4454-9c6b-e461d9f0a058 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.255846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb7d29-0944-4454-9c6b-e461d9f0a058 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.255846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb7d29-0944-4454-9c6b-e461d9f0a058 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.257536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 444/889] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:21.263904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c790e5-dc95-4007-a11e-ba1b8116a9dd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.264432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c790e5-dc95-4007-a11e-ba1b8116a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.264432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c790e5-dc95-4007-a11e-ba1b8116a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.269885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:21.283194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-12c790e5-dc95-4007-a11e-ba1b8116a9dd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:21.284405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-12c790e5-dc95-4007-a11e-ba1b8116a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:21.369179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 446/890] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:21.395225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb7d29-0944-4454-9c6b-e461d9f0a058 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:21.406884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad1a5-9577-44d2-aebd-af711bd12d55 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.407167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad1a5-9577-44d2-aebd-af711bd12d55 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.407447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ad1a5-9577-44d2-aebd-af711bd12d55 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.409121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 447/891] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:21.414686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232915ba-e90f-4fe0-a80b-3f2c169b442e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.415007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232915ba-e90f-4fe0-a80b-3f2c169b442e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.415284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232915ba-e90f-4fe0-a80b-3f2c169b442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.439709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-232915ba-e90f-4fe0-a80b-3f2c169b442e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:21.441309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-232915ba-e90f-4fe0-a80b-3f2c169b442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:21.517034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 448/892] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:21.518789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8d19e-a342-4498-850f-fde62622d76e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.519316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8d19e-a342-4498-850f-fde62622d76e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.519786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8d19e-a342-4498-850f-fde62622d76e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.521573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 449/893] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:21.527519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.528260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.528743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.531178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb7d29-0944-4454-9c6b-e461d9f0a058 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:21.552200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:21.553732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:21.572408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:21.573260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.573538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.573808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.586223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:21.594695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:21.595760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:21.596636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 445/894] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 25 14:27:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:21.597464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:21.601772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:21.624756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:21.642910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 450/895] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:21.644232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:21.712940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df27d-c07c-4b4e-b688-3e13fe8b5269 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:21.747841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd05e9-ad63-4b64-80be-1d0c6d92f3ed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:21.749627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:21.750514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.750822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.751099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.751766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:21.753159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:21.766665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:21.767771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:21.768609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 446/896] 10.222.0.32 () {56 vars in 1132 bytes} [Wed May 25 14:27:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:21.769431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:21.798559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:21.799287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.799565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.799832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.801055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:21.801055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:21.802231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:21.811349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 451/897] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:27:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:21.816890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:21.849651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:21.879088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:21.888293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62763e43-d596-44ae-a8e1-de380aad7af1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:21.928314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:21.928691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3cd5d-3995-4de0-982d-c2e059aec39f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:21.929002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.929282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.929556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:21.930319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:21.932226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:21.949308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 447/898] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:21 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:21.950000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:21.996914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:21.998028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:21.998509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:21.999007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.000795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:22.001350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.004172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.012775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 452/899] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:22.018516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.075884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fed693-da8f-4b80-9478-b01beb0935a9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.108095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:22.120455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.121303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.121585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.121862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.136431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.137950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.154536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf3639-fea6-4e92-8309-e1efaff13bc8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:22.157362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:22.158261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:22.159269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:22.159269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:22.159269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:22.159269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:22.159269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 448/900] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:22.167856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.174457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:22.203946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.206123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.206123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.206123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.206699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:22.208632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.208779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.215536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 453/901] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:22.225846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.296259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9048e-d063-4eea-8bd6-8a6f4e2101e0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.334775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.335636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.336016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.336452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.337256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:22.337995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.339297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.348601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6e730-548d-4c68-95b1-afd4d89df28c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.353072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 449/902] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:22.358438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.389894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.391096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.391584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.392072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.393214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] RBAC: Adding query filter params (interface=public, service_id=6acd7ba5eb9d40e7a997b335eea23578, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:22.393948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.395757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.402675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 454/903] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/endpoints?service_id=6acd7ba5eb9d40e7a997b335eea23578&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:22.409947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:22.412330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.470458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:22.475360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0dd74-711d-4754-8fee-3ad667800c98 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.511642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.512479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.512756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.513030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.520551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.522014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.531723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f25c2-48a2-498b-84dd-d6e4ce146f2a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-af116b92-7cf0-4fe1-8b43-6d8460ac0850 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:22.537095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:22.538038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:22.538979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 450/904] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:22.539939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f45156-898c-4e22-af72-ca960b37817e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.539939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f45156-898c-4e22-af72-ca960b37817e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.540124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f45156-898c-4e22-af72-ca960b37817e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.545965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 451/905] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:22.545965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f45156-898c-4e22-af72-ca960b37817e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.585826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71b45c-05b6-49f2-b463-32bd0afe6cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.586439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b45c-05b6-49f2-b463-32bd0afe6cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.586719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b45c-05b6-49f2-b463-32bd0afe6cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.587024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b45c-05b6-49f2-b463-32bd0afe6cf5 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.587729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71b45c-05b6-49f2-b463-32bd0afe6cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.589148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71b45c-05b6-49f2-b463-32bd0afe6cf5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.603111 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 455/906] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:27:22.604602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.604943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.605225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.624687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:22.625988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:22.677101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:22.684918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f45156-898c-4e22-af72-ca960b37817e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.712803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 456/907] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:22.740757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.745436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.746508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.747014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.747488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.748590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:22.749106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.751753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.780131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 452/908] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:22.781669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:22.790811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.863496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec2c5-e3d7-4737-9b14-2512c3ecc68d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.907887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.908834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.909118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.909391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:22.910168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:22.911739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:22.925557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:22.926584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:22.927439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 457/909] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:22.933012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:22.933160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da999955-223e-416f-996b-d6efdf5894a2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:22.972697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:22.973482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:22.973763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:22.974035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.006625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:23.063533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.065568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.065983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:23.071171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1598b71-940a-42e6-8ce2-479300a96b36 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:23.120125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:23.121450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:23.121833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:23.122196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.123081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:23.123468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.125705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.132913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 458/910] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:23.138163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:23.215103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 453/911] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:23.216106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:23.262480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf1863-aa00-459f-9eea-544a9afeacf5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:23.284729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:23.300857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:23.301604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:23.301819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:23.302088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.302841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:23.303152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.304551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.315007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 459/912] 10.222.0.32 () {56 vars in 1114 bytes} [Wed May 25 14:27:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:23.316000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:23.340411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a875b40-b39b-4d2d-9fc7-993599b74d9d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:23.363318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:23.388670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:23.390074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:23.390501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:23.390956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.406805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.409356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.434013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903f7a-8459-480a-88bd-cb58160ba694 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:23.435601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:23.436643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:23.437712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:23.437712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:23.437712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:23.437712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:23.440516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 454/913] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:23.448496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:23.481872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:23.482636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:23.482972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:23.483209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.483961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.486682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.519028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 460/914] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:23.618485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d098317-93da-4dc9-8642-64d21d7151b4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:23.631786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:23.667021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:23.668068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:23.668575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:23.669065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.670010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:23.670567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.672223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.690890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 455/915] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:23.693782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:23.799699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:23.816956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f37c26-354b-45a9-b9d4-23028bc16180 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:23.855738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:23.856823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:23.857282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:23.857708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:23.866173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:23.867938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:23.883953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:23.885157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:23.886224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:23.888527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 461/916] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:23.894514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:24.008035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebebde1-d094-472a-9add-1648521b9f67 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:24.046411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:24.047230 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.047509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.047781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.048943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:24.049249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:24.050782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6ef4a-a04d-4de5-b4bd-99388cac1538 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:24.067757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 456/917] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:24.078369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:24.188762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87bfe-744a-47dc-a2f4-ba0dc865c841 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:24.232109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:24.232791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.233071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.233344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.234205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] RBAC: Adding query filter params (role.id=d7930b987c3741b8ab8a3a0e213ae3d4, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=d61f6d6395f54f32adf1ba1ccdb6bc43) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:24.234513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:24.236036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f723e45-7f5b-4d43-8f9e-2c40275c48af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:24.243321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 462/918] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:24 2022] GET /identity/v3/role_assignments?user.id=d61f6d6395f54f32adf1ba1ccdb6bc43&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=d7930b987c3741b8ab8a3a0e213ae3d4 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:24.394689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d22e8-54a4-44cd-9398-bdc6ef9d3d3f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.395255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d22e8-54a4-44cd-9398-bdc6ef9d3d3f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.395255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d22e8-54a4-44cd-9398-bdc6ef9d3d3f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.397156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 457/919] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:24.403426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.404022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.404022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.422644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:24.424271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:24.426648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:24.501672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 463/920] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:24.527914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d22e8-54a4-44cd-9398-bdc6ef9d3d3f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:24.557642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:24.639072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d22e8-54a4-44cd-9398-bdc6ef9d3d3f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:24.676476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:24.677669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.678158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.678626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.679516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:24.681116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:24.693900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:24.694961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:24.695788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:24.702850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 458/921] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:24.705265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:24.817965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287ea27-e4d5-4f89-8b47-205f7f6c37b8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:24.858573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:24.859312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.859603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.859876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.860616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:24.860926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:24.862334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536cef5-ea54-4ca1-b332-a2dcdde5a2d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:24.868129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 464/922] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:24.874140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:24.933665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d176d-ee59-49f3-a6b1-03dad77b9e9b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.933865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d176d-ee59-49f3-a6b1-03dad77b9e9b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.934124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d176d-ee59-49f3-a6b1-03dad77b9e9b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.935881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 465/923] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:24.944847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:24.945213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:24.945460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:24.957808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:24.967357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:24.968757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:24.987141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:24.997592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800477b7-db81-4f24-81d3-c69d19d35e3b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.039672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.040979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.041473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.042063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.043039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:25.043753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.045501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.051132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 466/924] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:25.057302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 459/925] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:25.063885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:25.071337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:25.170532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b28c3-c90b-43cc-aa36-a125b6ba8230 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.183145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e7dc7-28f7-44c3-b427-adf06603a506 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.204425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:25.213260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.214377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.216267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.216955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.217977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] RBAC: Adding query filter params (interface=public, service_id=cb3c6045941a44069647d1ed8edebb41, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:25.218394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.220091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.230188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 467/926] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:25 2022] GET /identity/v3/endpoints?service_id=cb3c6045941a44069647d1ed8edebb41&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:25.236121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:25.237629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.238734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.239306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.239752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.240667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.242296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:25.256980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:25.258252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 14:27:25.259386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:25.262479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 460/927] 10.222.0.32 () {56 vars in 1132 bytes} [Wed May 25 14:27:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:25.269660 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:25.296696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:25.374056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1906e06-4121-4b06-af6f-16ca40b03f6c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.411682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe213f-0606-46a6-a0c1-0a0f8e1563d0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.435665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.437278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.437755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.438598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.439518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.441085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.449923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 468/928] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:25 2022] GET /identity/v3/services => generated 1497 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:27:25.466739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.467825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.468332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.468801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.469764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:25.470277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.472062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.478923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 461/929] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:25.487074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:25.563983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:25.610600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6b074-578b-49a9-aed4-6008d9c82ded None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.648826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.649950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.650465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.650972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.651941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:25.652441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.654091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c4934-f67a-45e5-8afd-9fadfb1de1e3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.660631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 469/930] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:25.669034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:25.823730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:25.827904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6502e-96ae-403e-8835-ece702141360 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:25.865547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821c13-2e39-41f7-8306-e568c95f9e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:25.866231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821c13-2e39-41f7-8306-e568c95f9e09 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.866649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821c13-2e39-41f7-8306-e568c95f9e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.866947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821c13-2e39-41f7-8306-e568c95f9e09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.867700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61821c13-2e39-41f7-8306-e568c95f9e09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:25.869178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61821c13-2e39-41f7-8306-e568c95f9e09 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:25.895910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 462/931] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:27:25.972183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79326af3-cc95-4b17-a58a-ad086713ada0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.972480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79326af3-cc95-4b17-a58a-ad086713ada0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.972729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79326af3-cc95-4b17-a58a-ad086713ada0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.974433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 470/932] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:25.980631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:25.980997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:25.981176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:25.999635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:26.000851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:26.088648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 463/933] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:26.112528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79326af3-cc95-4b17-a58a-ad086713ada0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:26.237228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79326af3-cc95-4b17-a58a-ad086713ada0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:26.263750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:26.276171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:26.276851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:26.277127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:26.277394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:26.278126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:26.279620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:26.281020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING keystone.server.flask.common [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 14:27:26.301232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:27:26.301232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 14:27:26.301232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [parameters: {'id': '628e4f16738042eeb7ca24f28d07811d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 14:27:26.301232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 628e4f16738042eeb7ca24f28d07811d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 628e4f16738042eeb7ca24f28d07811d. May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:27:26.309203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 14:27:26.310032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 25 14:27:26.311034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:27:26.311903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:27:26.312796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:27:26.313561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': '628e4f16738042eeb7ca24f28d07811d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:26.314495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:26.317277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 628e4f16738042eeb7ca24f28d07811d. May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:26.319414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 471/934] 10.222.0.32 () {60 vars in 1140 bytes} [Wed May 25 14:27:26 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:26.323837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:26.389798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:26.435395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a33a07-c995-47a6-bb45-693ddf576bf9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:26.480642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:26.481517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:26.481834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:26.482114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:26.492220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:26.493828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d5e84010-b6f7-45c7-9fbb-fd9cac6de4cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:26.509259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:26.510199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:26.511122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 464/935] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:26.516632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:26.634950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935682bb-8c42-4755-a499-4c5ec284c020 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:26.669869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:26.670451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:26.670727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:26.671026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:26.671787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:26.672087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:26.674061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:26.680753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091e322-8f07-4bc1-a430-d7d9ee958310 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:26.681411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091e322-8f07-4bc1-a430-d7d9ee958310 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:26.681886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091e322-8f07-4bc1-a430-d7d9ee958310 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:26.683776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 465/936] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:26.689513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:26.690064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:26.690554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:26.696413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 472/937] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:26.709164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:26.710991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:26.796217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 466/938] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:26.822625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:26.879061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:26.943318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84467c9d-dabf-4bd2-b7bb-303620020550 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:26.978609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:26.979849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:26.980328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:26.980792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:26.991943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:27.000673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:27.001571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:27.002382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:27.005143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 473/939] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 25 14:27:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:27.013147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:27.070031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:27.115768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bcc9b-5d99-4fa4-ad50-1cb569f73f36 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:27.151411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:27.152092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.152487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.152765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.153514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:27.153817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:27.155238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:27.163928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 467/940] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:27:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:27.169690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:27.274889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:27.278664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ceba25-771d-4b4e-8b7b-d9609fe79f99 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:27.314616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:27.315455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.315731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.315999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.327485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:27.329056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:27.346848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:27.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:27.348676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:27.348676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:27.348676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:27.348676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:27.348676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 474/941] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:27.354144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:27.464293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e18f1-31a1-47ff-89b2-12772e7a3c82 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:27.486227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:27.500840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:27.501569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.501856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.502144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.502915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:27.503348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:27.504738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:27.518049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 468/942] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:27.523278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:27.625897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c18542-0de4-41c2-8f00-44e22d64b2bd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:27.659923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:27.660736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.661013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.661284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.669694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:27.671167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:27.686630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:27.688895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:27.691318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 475/943] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:27:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:27.696235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:27.808302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4494-8b7f-43b8-ad11-c13475a9e7e2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:27.849048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:27.849737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.850021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.850297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.851215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:27.851529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:27.853041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66b1b9-b875-4bce-9682-6046f2394bc7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:27.871797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 469/944] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 25 14:27:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:27.877447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:27.907181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:27.918209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c915f2-5bfe-4ba2-b00d-1509b4d96170 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.918559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c915f2-5bfe-4ba2-b00d-1509b4d96170 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.918773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c915f2-5bfe-4ba2-b00d-1509b4d96170 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.920401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 470/945] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:27.926177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:27.926461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:27.926742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:27.943901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:27.945107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:27.982293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e697-1dc6-484a-ba86-367d18f39ec0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:27.995131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.017663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.019055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.019055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.019055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.023188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 471/946] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:28.032397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.032397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.046332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:28.050017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:28.050957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:28.051824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:28.051824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:28.051824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:28.051824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:28.051824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 476/947] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:28.056975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.126742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.161481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02406cc2-a099-4794-883a-e5312f475a31 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.170693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ce07e-a92f-4a05-8132-f5e1973bb4ff None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.200827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.201668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.201961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.202344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.203171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.204626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.208100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.208784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.209068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.209339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.210078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:28.210384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.211799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:28.219520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:28.220512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:28.221352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:28.224286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 472/948] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:28.224706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 477/949] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:28.234839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.235044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.272479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.343015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67245c44-daa1-4010-bf55-6e74a0be3b02 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.350778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0e2d9-495d-401d-b2bf-28fe2bb0e065 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.368838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.382017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.382967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.383641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.383924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.384964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:28.385272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.386677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda51ad-8672-4acf-a29a-cdf502944f0f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.388179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.388991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.389267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.389534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.394908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 478/950] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:28.396302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada52a5-2262-4d24-957c-585707a05e4d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.396784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada52a5-2262-4d24-957c-585707a05e4d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.396951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada52a5-2262-4d24-957c-585707a05e4d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.398192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 479/951] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:28.398972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada52a5-2262-4d24-957c-585707a05e4d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.404483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.406176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-21b9ad48-17c0-4199-bbfd-37bdcc38f765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:28.421181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:28.422259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:28.423807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:28.424238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 473/952] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:28.425445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.425725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.425993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.448051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:28.449516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:28.516529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 474/953] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:28.517186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.524792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada52a5-2262-4d24-957c-585707a05e4d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.543247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.562421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.563127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.563432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.563672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.564384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:28.564685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.566054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.571134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 480/954] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:28.571968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.627510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.649910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c7cfc-ae15-4240-bbb7-f9aad8b7cc68 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.680554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528aef83-814d-4383-9276-a1b0deb4e231 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.686627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.687328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.687601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.687895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.689160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:28.689465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.691445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.704932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 475/955] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:28.705751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.723914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.724729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.725009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.725280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.725932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.727332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:28.739571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:28.740688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:28.741537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:28.745837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 481/956] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:28.746991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.790203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.813672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078803c-7545-471e-8ba0-00fca4d3f9e8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.857651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8629a2-757c-40c3-9419-62ec2f2003dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:28.858324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.859046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.859630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.859907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.861725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.863195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.876931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:28.889640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 476/957] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:28.890534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:28.909894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a89b3c-1f04-4139-88a5-1b7626f339fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:28.911391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a89b3c-1f04-4139-88a5-1b7626f339fd None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:28.911907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a89b3c-1f04-4139-88a5-1b7626f339fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:28.912530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a89b3c-1f04-4139-88a5-1b7626f339fd None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/d61f6d6395f54f32adf1ba1ccdb6bc43/roles/d7930b987c3741b8ab8a3a0e213ae3d4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:28.915390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a89b3c-1f04-4139-88a5-1b7626f339fd None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=d61f6d6395f54f32adf1ba1ccdb6bc43, role_id=d7930b987c3741b8ab8a3a0e213ae3d4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:28.917248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a89b3c-1f04-4139-88a5-1b7626f339fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:28.941567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 482/958] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:28 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/d61f6d6395f54f32adf1ba1ccdb6bc43/roles/d7930b987c3741b8ab8a3a0e213ae3d4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:27:29.024128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf91dc-80fa-482b-80ef-95376d9ea2d8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:29.050332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99dc57-a5ca-480f-9303-0bd6b8c1e93e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.050559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99dc57-a5ca-480f-9303-0bd6b8c1e93e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.050877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99dc57-a5ca-480f-9303-0bd6b8c1e93e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.052682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 483/959] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:29.058898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.059251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.059445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.069816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:29.070578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.070899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.071175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.072022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] RBAC: Adding query filter params (name=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:29.072330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:29.074861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:29.076243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:29.079344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:29.081226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:29.082124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 477/960] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:29.089591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:29.185259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 484/961] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:29.211725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:29.214846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ea4b0-a9e0-4593-9beb-94b82c00d8ca None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:29.260671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:29.261450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.261803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.262142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.262959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:29.263332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:29.265255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:29.272556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 478/962] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:29.319306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836397a-9bda-4013-b414-e836ba5cc38f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:29.321436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:29.360785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:29.361853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.362327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.362816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.363762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:29.365420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:29.380654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:27:29.380654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 14:27:29.380654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [parameters: {'user_id': '83ac2ee277d145acaf69b49ef6930696', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 14:27:29.380654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 25 14:27:29.389596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:27:29.391737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 14:27:29.392767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 14:27:29.393636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 14:27:29.394507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 14:27:29.396442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'user_id': '83ac2ee277d145acaf69b49ef6930696', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:29.397274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 14:27:29.398084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 14:27:29.400014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 14:27:29.400014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 14:27:29.400014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 14:27:29.400014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 14:27:29.400014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 14:27:29.400014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:29.401949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 485/963] 10.222.0.32 () {60 vars in 1131 bytes} [Wed May 25 14:27:29 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:29.410862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:29.482350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:29.525031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3e586-e711-4648-a50c-a0f9d0a1e6b4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:29.560708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:29.561890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.562385 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.562869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.571050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:29.572786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-30e939c4-7c4d-4cb0-9f3e-483cc58687a5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:29.587162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:29.588183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 14:27:29.589110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:29.591845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 479/964] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:29.597671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:29.705479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e228a7b-245a-402f-92fb-5ed0b71bfb9b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:29.746997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:29.747592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:29.747870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:29.748138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:29.748962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:29.749264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:29.750937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab2de3-b6cb-409d-a386-cd1964884c0f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:29.789962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 486/965] 10.222.0.32 () {56 vars in 1101 bytes} [Wed May 25 14:27:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:31.204748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1de5-0a63-49cc-b342-e81d1cf06190 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.205303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1de5-0a63-49cc-b342-e81d1cf06190 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.205303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f1de5-0a63-49cc-b342-e81d1cf06190 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.210382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 480/966] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:31.212272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.212691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.212766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.230568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:31.230951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60a469-3fa4-4815-ab5b-7e4e86da038b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.231184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60a469-3fa4-4815-ab5b-7e4e86da038b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.231460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60a469-3fa4-4815-ab5b-7e4e86da038b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.232104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:31.232970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 481/967] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:31.237929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.238168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.238469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.251432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.256576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:31.258217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:31.304143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 487/968] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:31.327102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.336995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 482/969] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:31.360519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.417540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.437881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900354be-1f0c-4fff-b25c-fbc95c9a5b68 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.473327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1f06e-8180-4b29-90d2-ef1d197df069 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.474523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.475397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.475684 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.475948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.476610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:31.478069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:31.490465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:31.491667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:31.497486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 488/970] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:31.499608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.515181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1791530-a7a4-4949-8de2-d12737834a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.516010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1791530-a7a4-4949-8de2-d12737834a3e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.516287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1791530-a7a4-4949-8de2-d12737834a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.516556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1791530-a7a4-4949-8de2-d12737834a3e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.519386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.527099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1791530-a7a4-4949-8de2-d12737834a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e1791530-a7a4-4949-8de2-d12737834a3e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:31.535281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:31.536211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:31.537054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 483/971] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:31.537749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246702d6-3511-446b-8685-8e6ba0f70be2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.538266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246702d6-3511-446b-8685-8e6ba0f70be2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.538266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246702d6-3511-446b-8685-8e6ba0f70be2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.539666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 484/972] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:31.543575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246702d6-3511-446b-8685-8e6ba0f70be2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.599241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea09fd-1035-4aef-8082-9417c981d323 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.631903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.632562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.632844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.633114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.633811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:31.634273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:31.635640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab50df4-6b4d-4080-8d3c-c590735d04c4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:31.640588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246702d6-3511-446b-8685-8e6ba0f70be2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.640709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 489/973] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:31.644075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.644360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.644633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.657123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.662021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:31.663240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:31.680757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.681479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.681759 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.682033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.682815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:31.683126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:31.684541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:31.692826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 485/974] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:31.693615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.728475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 490/975] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:31.729243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.740000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.798640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5475a-d8a8-495b-9da0-281d189fbc20 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.831036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.831790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b92968-6f96-45ed-8174-1f8ad65232f4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.831905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.832097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.832368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.833176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:31.833370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:31.834738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:31.839659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 486/976] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:31.840454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.869382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.870188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.870468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.870738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.879460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.880622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:31.882221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:31.900031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:31.900915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:31.901772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:31.901772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:31.901772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:31.901772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:31.901772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 491/977] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:31.902058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:31.941739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5dfe9-180e-4d93-86e6-c84579e608a0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:31.964475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:31.975420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:31.976094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:31.976373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:31.976642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:31.977366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:31.978832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:31.992635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 487/978] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:31 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:31.993543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:32.004408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06a5b2-0eec-4c2c-9e6a-267b531c87f7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:32.060126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:32.060812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:32.061089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:32.061359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:32.062180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] RBAC: Adding query filter params (interface=admin, service_id=cb3c6045941a44069647d1ed8edebb41, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:32.062548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:32.063994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:32.074953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 492/979] 10.222.0.32 () {56 vars in 1249 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/endpoints?service_id=cb3c6045941a44069647d1ed8edebb41&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:32.076522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:32.128840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:32.134689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46180c17-3404-4b7f-8583-5cb45032e9dd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:32.179649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:32.180332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:32.180614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:32.180922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:32.181668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:32.181976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:32.183498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:32.187600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425279f8-945d-47e1-bd37-ee9ca277f507 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:32.196230 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 488/980] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:32.201861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:32.212744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:32.227804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:32.228475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:32.228790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:32.229121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:32.229832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:32.231340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:32.242778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 493/981] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:32 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:27:32.314005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24e55-ad3c-4ac0-9796-3d3fd6d52fff None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:32.352616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:32.353521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:32.353803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:32.354052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:32.362795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:32.364292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:32.380699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:32.381715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:32.383526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 489/982] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:27:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:32.388236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:32.498587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55a12a-9f6a-4c71-88bd-2a75db1d7b42 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:32.534157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:32.534844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:32.535113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:32.535376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:32.536215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:32.536515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:32.537930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:32.555929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 494/983] 10.222.0.32 () {56 vars in 1149 bytes} [Wed May 25 14:27:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:32.561596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:32.582645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:32.670670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4db85c-6921-4ff3-930b-68e72549ebd1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:32.706142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:32.706824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:32.707108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:32.707379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:32.780135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:32.781962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:32.804412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:32.890382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 490/984] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:32 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:32.896356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:32.944386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:33.000522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b63f0-fb55-48ae-895b-2eea0067081e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:33.035958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:33.036664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:33.036938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:33.037203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:33.048299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:33.049747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:33.068778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:33.069785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:33.075520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:33.075520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:33.075520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:33.075520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:33.075520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 495/985] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:33.077644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:33.184305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f7693-98f3-4f4c-a559-5a71c6820fa1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:33.201555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:33.219600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:33.220266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:33.220544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:33.220817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:33.221539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:33.221841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:33.223266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:33.239050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 491/986] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:33.240041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:33.336542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:33.342820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9cb28-90ec-4f37-8103-056749a51a57 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:33.380439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:33.381337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:33.381660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:33.381936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:33.388983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:33.390412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:33.405469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:33.406350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:33.411377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 496/987] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:33.414310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:33.517212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92318e87-6715-4099-9ae1-6f7a156d9655 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:33.551521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:33.552184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:33.552457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:33.552726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:33.553505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:33.553808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:33.555275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577662a0-0984-4d0f-a8f1-09946c9dab99 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:33.567475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 492/988] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:33.573376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:33.672408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6bc6d-1959-426d-8b1c-9de05674f85f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:33.705763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:33.706427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:33.706698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:33.706992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:33.707818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] RBAC: Adding query filter params (role.id=d7930b987c3741b8ab8a3a0e213ae3d4, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=d61f6d6395f54f32adf1ba1ccdb6bc43) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:33.708132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:33.709553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2da89-06f1-4e5f-812e-1c42ce245662 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:33.715595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 497/989] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:33 2022] GET /identity/v3/role_assignments?user.id=d61f6d6395f54f32adf1ba1ccdb6bc43&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=d7930b987c3741b8ab8a3a0e213ae3d4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:34.229943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541de6-17df-4c92-8344-dde18bbd321e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.230334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541de6-17df-4c92-8344-dde18bbd321e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.230544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541de6-17df-4c92-8344-dde18bbd321e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.231972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 493/990] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:34.237496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530470-798c-4ee1-9c67-943465893db6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.237789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530470-798c-4ee1-9c67-943465893db6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.238061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530470-798c-4ee1-9c67-943465893db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.255596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-44530470-798c-4ee1-9c67-943465893db6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:34.256819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-44530470-798c-4ee1-9c67-943465893db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:34.273396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:34.331879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 498/991] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:34.354053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541de6-17df-4c92-8344-dde18bbd321e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.396473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:34.454854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a254db-daab-43a4-b4b2-0079236b7d1e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.455169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a254db-daab-43a4-b4b2-0079236b7d1e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.455549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a254db-daab-43a4-b4b2-0079236b7d1e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.456928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 499/992] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:34.459558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541de6-17df-4c92-8344-dde18bbd321e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:34.462249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.462590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.462945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.479490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:34.480715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:34.495850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:34.497026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.497482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.497985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.498844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:34.500526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:34.513145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:34.514139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:34.515120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 494/993] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:34.520458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.553669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 500/994] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:34.575856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.629251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffb1fa-f625-4fd0-ba92-b1a4d5702b78 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:34.630943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:34.655892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:34.668018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:34.669131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.669906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.670617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.671683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] RBAC: Adding query filter params (name=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:34.672776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:34.674641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:34.675289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6d72-962a-4870-9d8f-7fd2690023d6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:34.682503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 495/995] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:34.687114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.712493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:34.713276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.713554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.713818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.714466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:34.715869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:34.727542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:34.728553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:34.729557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 501/996] 10.222.0.32 () {56 vars in 1126 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:34.734671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.800027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3508cd-d816-43d7-9d11-793a12daafee None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:34.835706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:34.836373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.836643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.836941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.837806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:34.838109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:34.839505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:34.849182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 496/997] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:34.849422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a31e1-440f-4830-a3d2-e7ac724093e9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:34.850412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.880443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:34.887255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:34.887934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:34.888211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:34.888479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:34.889298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:34.889472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:34.890869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:34.901575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 502/998] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:34.903329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:34.915344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:34.961506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5534a-2cfd-4438-98a2-cbe755ea9ec6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:35.001575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:35.002143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:35.002430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:35.002690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:35.003512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] RBAC: Adding query filter params (interface=public, service_id=c9fa3788273d4b65b82fb4fce0d03771, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:35.003823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:35.005351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:35.009245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2765a-94e3-488e-8109-369d20bd7386 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:35.014805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 497/999] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/endpoints?service_id=c9fa3788273d4b65b82fb4fce0d03771&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:35.015873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:35.051999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:35.052715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:35.052994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:35.053262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:35.053960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:35.054260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:35.055672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:35.061079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 503/1000] 10.222.0.32 () {56 vars in 1122 bytes} [Wed May 25 14:27:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:35.066723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:35.114440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:35.122201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b13c80-50d2-44f4-b53c-1512e47a18f9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:35.140886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:35.158478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7272bd-b1e9-4f25-b9dd-09140472e082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:35.159191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7272bd-b1e9-4f25-b9dd-09140472e082 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:35.159460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7272bd-b1e9-4f25-b9dd-09140472e082 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:35.159732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7272bd-b1e9-4f25-b9dd-09140472e082 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:35.160404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7272bd-b1e9-4f25-b9dd-09140472e082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:35.161775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7272bd-b1e9-4f25-b9dd-09140472e082 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:35.168437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 498/1001] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:35 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:27:35.172686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249dacfd-cee5-4ff6-83d7-9521f1dcaa95 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:35.209700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295b33c-53d3-4c34-8bd8-9dfdfe568b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:35.210380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295b33c-53d3-4c34-8bd8-9dfdfe568b76 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:35.210655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295b33c-53d3-4c34-8bd8-9dfdfe568b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:35.210956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295b33c-53d3-4c34-8bd8-9dfdfe568b76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:35.211707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295b33c-53d3-4c34-8bd8-9dfdfe568b76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:35.213177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295b33c-53d3-4c34-8bd8-9dfdfe568b76 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:35.238804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 504/1002] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:27:35.933314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854a4bb-5e09-499c-a154-7fe5f0a3a7e2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:35.933860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854a4bb-5e09-499c-a154-7fe5f0a3a7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:35.933860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854a4bb-5e09-499c-a154-7fe5f0a3a7e2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:35.935211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 499/1003] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:35.940673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:35.941107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:35.941181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:35.958258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:35.959467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:36.034851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 505/1004] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:36.057824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854a4bb-5e09-499c-a154-7fe5f0a3a7e2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:36.157697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854a4bb-5e09-499c-a154-7fe5f0a3a7e2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:36.193070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:36.193855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:36.194129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:36.194394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] PATH_INFO: `/v3/services/image` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:36.195057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:36.196474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:36.207935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:36.208919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:36.214831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 500/1005] 10.222.0.32 () {56 vars in 1105 bytes} [Wed May 25 14:27:36 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:36.217228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:36.275975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:36.326849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af6537-ed37-4061-9036-812611e98b70 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:36.364625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:36.365613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:36.366045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:36.366661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:36.367551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:36.368003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:36.369614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cac74a-c49c-41a6-be00-305a17dbeb9f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:36.380581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 506/1006] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:36.382304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:36.409959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:36.492247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4115a41-4eba-4270-9ade-a4aa82c95b61 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:36.528036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:36.528985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:36.529415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:36.529873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:36.530815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] RBAC: Adding query filter params (type=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:36.531297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:36.532836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ac62f-ddff-4615-b071-7dbdd82c4259 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:36.538049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 501/1007] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:37.383110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ceafb-3cea-4ac4-aa97-0909064b9ca8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.383661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ceafb-3cea-4ac4-aa97-0909064b9ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.383661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ceafb-3cea-4ac4-aa97-0909064b9ca8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.385417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 507/1008] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:37.390530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.390986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.391152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.408167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:37.409346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:37.441048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f105464-7681-4112-8587-3e9d6add56f1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.441354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f105464-7681-4112-8587-3e9d6add56f1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.441565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f105464-7681-4112-8587-3e9d6add56f1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.442964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 508/1009] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:37.447896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.448334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.448781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.465063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:37.466452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:37.479700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 502/1010] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:37.501893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:37.518732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:37.540828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 509/1011] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:27:37.564599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:37.605751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cae202-5c49-4bc7-a133-c8d2c30605b2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:37.619085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:37.641209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:37.641931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.642625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.642930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.643584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:37.645312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:37.657573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:37.663968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:37.664857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 503/1012] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:37.665665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:37.677274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186a014-70bc-4aa1-af7b-75a34fb09e28 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:37.713176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:37.713969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.714242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.714510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.715702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:37.717137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:37.719126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 510/1013] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:37.724725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:37.754569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:37.775621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0f426-adec-4512-9b0b-e9a33f75e309 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:37.811737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:37.812514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.812822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.813059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.813768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:37.814074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:37.815478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:37.820551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 504/1014] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:37.825994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:37.834060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c7f32-5ee8-4268-bea4-af069785acaa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:37.855298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:37.871618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:37.872276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.872601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.872848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.873507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:37.874952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:37.910872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 511/1015] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:37 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:37.938340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893cb62d-e687-4309-b932-1420fddbd69e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:37.974601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:37.975359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:37.975642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:37.975915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:37.976649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:37.976955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:37.978472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6771-9aaa-4d74-b0bb-5d44844709cf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:37.989254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 505/1016] 10.222.0.32 () {56 vars in 1112 bytes} [Wed May 25 14:27:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:37.991031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:38.100322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cf00b-e49b-4484-9b63-1f8bf6076e95 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:38.143222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f87e3-c1b8-41f4-a66c-2828e766e333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:38.143882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f87e3-c1b8-41f4-a66c-2828e766e333 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:38.144161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f87e3-c1b8-41f4-a66c-2828e766e333 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:38.144433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f87e3-c1b8-41f4-a66c-2828e766e333 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:38.145181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f87e3-c1b8-41f4-a66c-2828e766e333 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:38.146535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f87e3-c1b8-41f4-a66c-2828e766e333 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:38.168633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 512/1017] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:37 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:27:38.757124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9cc72-8093-4cee-b222-bb16cdadf7b5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:38.757716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9cc72-8093-4cee-b222-bb16cdadf7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:38.757716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9cc72-8093-4cee-b222-bb16cdadf7b5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:38.759318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 506/1018] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:38.764804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad4877-4b06-4bf7-9665-ce1bc65823df None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:38.765240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad4877-4b06-4bf7-9665-ce1bc65823df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:38.765337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad4877-4b06-4bf7-9665-ce1bc65823df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:38.783421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f9ad4877-4b06-4bf7-9665-ce1bc65823df None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:38.784633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9ad4877-4b06-4bf7-9665-ce1bc65823df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:38.812478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:38.866804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 513/1019] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:38 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:38.888826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9cc72-8093-4cee-b222-bb16cdadf7b5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:38.900556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:38.996077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9cc72-8093-4cee-b222-bb16cdadf7b5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:39.031534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6f8e-7ff0-4cb4-9a44-fd64c11fd539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:39.032115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6f8e-7ff0-4cb4-9a44-fd64c11fd539 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:39.032388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6f8e-7ff0-4cb4-9a44-fd64c11fd539 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:39.032653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6f8e-7ff0-4cb4-9a44-fd64c11fd539 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:39.033407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b6f8e-7ff0-4cb4-9a44-fd64c11fd539 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:39.034787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b6f8e-7ff0-4cb4-9a44-fd64c11fd539 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:39.058781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 507/1020] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:27:38 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:27:40.159677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc346fd2-2205-4c53-a5e5-d95424d46a0d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.160273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc346fd2-2205-4c53-a5e5-d95424d46a0d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.160273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc346fd2-2205-4c53-a5e5-d95424d46a0d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.161620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 514/1021] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:40.167997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089a00a-d7f2-4494-8903-1550b5493b31 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.168549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089a00a-d7f2-4494-8903-1550b5493b31 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.168549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089a00a-d7f2-4494-8903-1550b5493b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.186609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b089a00a-d7f2-4494-8903-1550b5493b31 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:40.188085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b089a00a-d7f2-4494-8903-1550b5493b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:40.257828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:40.272723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 508/1022] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:40.297037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc346fd2-2205-4c53-a5e5-d95424d46a0d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:40.402619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc346fd2-2205-4c53-a5e5-d95424d46a0d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:40.437487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:40.438490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.438490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.438830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.440044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:40.440173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca973b-a8f6-4349-b8a6-17cfc4ea3ac5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.440412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca973b-a8f6-4349-b8a6-17cfc4ea3ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.440680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca973b-a8f6-4349-b8a6-17cfc4ea3ac5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.441477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:40.442227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 509/1023] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:40.443252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 515/1024] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:40.447016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.447303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.447576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.448743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:40.465217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:40.466364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:40.476840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:40.536615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 510/1025] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:40.551894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee110496-8b89-45e8-85d3-4a2f679a7e71 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:40.559713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:40.588663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:40.589345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.589621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.589890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.590600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:40.592069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:40.622793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 516/1026] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:27:40 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:40.677909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af18f6d-a51a-420a-b5f1-4782ac394f36 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:40.705207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:40.718065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:40.718930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.719208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.719475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.731273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:40.739565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:40.740531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:40.741387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 511/1027] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 25 14:27:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:40.747389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:40.844273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:40.868788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3a3db-fd99-42f4-b6e8-4e79ba6cb54c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:40.904375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:40.905084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:40.905367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:40.905639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:40.906400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:40.906702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:40.908202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:40.917214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 517/1028] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:27:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:40.923267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.028668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedd5da-cf4c-4a96-bc73-0ff32403f968 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.063192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.063998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.064280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.064554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.075332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.076798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:41.094070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:41.095160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:41.096535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:41.096535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:41.096535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:41.096535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:41.096535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 512/1029] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:41.101796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.207214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36bb1d-9c78-4240-b36f-650e024730fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.240883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.241553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.241851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.242100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.242869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:41.243179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.244552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ada7-102f-4398-bae9-27f4761f48ee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.254167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:41.263104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 518/1030] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:41.264336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.287800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca009d8-9156-4c24-acc6-eb8f5ef5d204 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.288092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca009d8-9156-4c24-acc6-eb8f5ef5d204 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.288267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca009d8-9156-4c24-acc6-eb8f5ef5d204 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.289732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 519/1031] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:41.295023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.295303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.295570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.302352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:41.312939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:41.314112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:41.368905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e190a-0214-4f24-a84c-0c5f87d6add6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.386361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 520/1032] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:41.405125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.405911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.406200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.406466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.408400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.414227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.415808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:41.430795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:41.431780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:41.432739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:41.436782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 513/1033] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:41.437650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.511937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ca366-62e0-4041-9bad-87fc437d34d1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.533871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:41.540547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76299830-a76b-4524-97a7-9eeed6f8fe6f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.545981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:41.553992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.554813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.555092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.555408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.556062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.557426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:41.569138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:41.570117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:41.571010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 521/1034] 10.222.0.32 () {56 vars in 1105 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:41.576457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.582364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.583250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.583671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.584091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.585097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:41.585562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.587305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.611171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 514/1035] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:41.612264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.689409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2b979-9f96-4826-be83-0a4641cfeaa3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.733595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.734306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.734581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.734882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.735600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:41.735899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.737261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.739297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8b99-7c9a-4cdb-9d7d-af423727091b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.742646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 522/1036] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:41.749305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.807967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.808812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.809872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.809872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.831137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:41.839499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:41.877287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2990-9d58-43d0-b3fd-6de68ece4170 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:41.884342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.885810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.911750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:41.912439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:41.912712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:41.912977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:41.913673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] RBAC: Adding query filter params (type=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:41.913972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:41.915432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:41.920501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 523/1037] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:41.926520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:41.998499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 515/1038] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:42.000196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:42.031973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323826-415e-4344-9870-ffcbe192aa15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.054415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:42.058285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:42.069394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.070236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.070688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.071201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.072125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] RBAC: Adding query filter params (interface=public, service_id=0f9fea63031040d78182958fd9b62a55, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:42.072577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.074095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.079665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 524/1039] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/endpoints?service_id=0f9fea63031040d78182958fd9b62a55&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:42.084895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:42.109507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c7eab-8ed8-4ed2-b792-adaa1c50c2e4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.145260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.146049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.146325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.146598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.156846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.158285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:42.176763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:42.177781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:42.178703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:42.178703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:42.178703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:42.178703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:42.178703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 516/1040] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:42.183635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:42.189421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654fdb9-d154-4678-821c-33a413397546 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.229181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.229833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.230109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.230376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.231109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.232475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.250785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 525/1041] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:27:42 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:27:42.304325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b4358-d517-471d-8414-837b0af4de3f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.313408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:42.346739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.347449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.347724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.347994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.348742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:42.349041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.350473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.363359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 517/1042] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:42.369088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:42.422653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:42.477448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdead9e5-e1ae-4cf4-9cdf-83f2e73720f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.510576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.511399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.511673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.511941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.518794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.520250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:42.534450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:42.535568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:42.536456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 526/1043] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:42.541546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:42.643962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf84b4f-c82b-4106-8c17-11244c6b408f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.677712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.678367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.678642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.678939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.679728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:42.680030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.681494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de579af8-0c4b-41b1-914e-5b8653721ca4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.693522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 518/1044] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:42.699502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:42.819019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b57a9-eb7e-42e8-a2ff-57f0f594e5ab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:42.830836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:42.856325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:42.856979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.857253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.857520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.858345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] RBAC: Adding query filter params (role.id=d7930b987c3741b8ab8a3a0e213ae3d4, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:42.858650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:42.860102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad04b3-1cd4-4d5a-894f-6945ed94e1b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:42.866533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 527/1045] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:42 2022] GET /identity/v3/role_assignments?user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=d7930b987c3741b8ab8a3a0e213ae3d4 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:42.936244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d01d9-b8b6-4c6d-b801-95489326ae57 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.936440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d01d9-b8b6-4c6d-b801-95489326ae57 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.936686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d01d9-b8b6-4c6d-b801-95489326ae57 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.938315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 519/1046] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:42.943981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:42.944282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:42.944521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:42.963214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:42.964422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:43.040233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 528/1047] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:43.063434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d01d9-b8b6-4c6d-b801-95489326ae57 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:43.091841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:43.171007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d01d9-b8b6-4c6d-b801-95489326ae57 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:43.206195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:43.206941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:43.207220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:43.207489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:43.217790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:43.225355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:43.226248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:43.227118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 520/1048] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:43.234380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:43.331809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a9951-9ccb-46c0-ab23-1a24f0b9ab6f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:43.334143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:43.368649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:43.369614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:43.370042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:43.370461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:43.371479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:43.372047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:43.373647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:43.384124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 529/1049] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:43.389636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:43.479188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:43.495798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd65089-53ad-4f18-b539-8ed13a0dda3a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:43.529526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:43.530406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:43.530680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:43.530977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] PATH_INFO: `/v3/users/025ea35f75b74da3b563acfbb8cf7b49` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:43.544493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] RBAC: Authorizing `identity:get_user(user_id=025ea35f75b74da3b563acfbb8cf7b49)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:43.545949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:43.552511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 521/1050] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:43 2022] GET /identity/v3/users/025ea35f75b74da3b563acfbb8cf7b49 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:43.553875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:43.657762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b9423-b5c8-4316-bf59-eca7e1400382 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:43.693879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:43.694576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:43.694868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:43.695165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:43.729879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:43.765170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:43.766864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:43.877582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 530/1051] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:27:43.883373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:43.986796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74965dca-d58c-4e9a-81e3-93a23879eb64 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:44.020893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bed5e-3ea1-4c85-8fbe-0cf15e1b41ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:44.021761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bed5e-3ea1-4c85-8fbe-0cf15e1b41ea None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:44.022038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bed5e-3ea1-4c85-8fbe-0cf15e1b41ea None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:44.022341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bed5e-3ea1-4c85-8fbe-0cf15e1b41ea None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:44.023475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bed5e-3ea1-4c85-8fbe-0cf15e1b41ea None admin] RBAC: Authorizing `identity:get_project(project_id=c6726fa6092b4f76b511ceeee08afe06)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:44.025022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bed5e-3ea1-4c85-8fbe-0cf15e1b41ea None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:44.027106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 522/1052] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:27:43 2022] GET /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:44.032803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:44.098228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:44.133324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442abf3a-4e2b-4d47-b6f7-c768591f63a1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:44.166671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:44.167353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:44.167628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:44.167895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:44.168723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=c6726fa6092b4f76b511ceeee08afe06, user.id=025ea35f75b74da3b563acfbb8cf7b49) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:44.169028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:44.170446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f04d1-c6f3-43fc-aaaf-14793dedb915 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:44.182774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 531/1053] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:44 2022] GET /identity/v3/role_assignments?user.id=025ea35f75b74da3b563acfbb8cf7b49&scope.project.id=c6726fa6092b4f76b511ceeee08afe06&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:44.478172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2fcba-18a1-4fe5-8cf3-cb8fcc185546 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:44.478745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2fcba-18a1-4fe5-8cf3-cb8fcc185546 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:44.478745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2fcba-18a1-4fe5-8cf3-cb8fcc185546 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:44.480214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 523/1054] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:44.485559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:44.485984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:44.486058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:44.503132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:44.504316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:44.576679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 532/1055] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:44.599728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2fcba-18a1-4fe5-8cf3-cb8fcc185546 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:44.615413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:44.707205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2fcba-18a1-4fe5-8cf3-cb8fcc185546 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:44.742376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:44.743190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:44.743468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:44.743738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] PATH_INFO: `/v3/services/image` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:44.744427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:44.745844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:44.757553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:44.758622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:44.764596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 524/1056] 10.222.0.32 () {56 vars in 1105 bytes} [Wed May 25 14:27:44 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:44.767514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:44.834938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:44.882504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80676e-c564-4f61-905c-b0866e570fca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:44.918906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:44.919576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:44.919850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:44.920115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:44.920818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:44.921112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:44.922841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c5ee0-6dac-4f48-b4cf-3c13b45de16b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:44.927953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 533/1057] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:44.933531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.013021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:45.035670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f2aa-86b8-485c-a38d-b958f2b577cf None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.035944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f2aa-86b8-485c-a38d-b958f2b577cf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.036214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f2aa-86b8-485c-a38d-b958f2b577cf None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.037596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 534/1058] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:45.039104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7b428-cf99-4875-9798-ae3ee2ba9154 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:45.043123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.043406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.043679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.061839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:45.063072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:45.076396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:45.077057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.077330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.077596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.078294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] RBAC: Adding query filter params (type=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:45.078593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:45.079998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:45.089321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 525/1059] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:45.090435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.132572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 535/1060] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:45.154356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.196870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb57447-994c-46d2-8acc-5b83b1ac93ae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:45.209733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:45.233643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:45.234600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.234930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.235354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.236210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:45.238256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:45.247810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:45.256272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9e299-24a5-4e43-a00f-d640e94a1b56 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:45.262542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 526/1061] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:27:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:45.316889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:45.317681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.317931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.318219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.330813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:45.339388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:45.340484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:45.341397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 536/1062] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 25 14:27:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:45.348567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.456852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7ae2f-9c28-49da-80dd-af6d8b95a51a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:45.493589 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:45.494361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.494547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.494842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.495587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:45.495893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:45.497259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:45.505663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 527/1063] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:27:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:45.511930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.613789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a78d-da18-425c-970a-6ef52ac7ef4f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:45.624099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:45.649373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:45.650219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.650471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.650777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.661755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:45.663246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:45.680844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:45.681791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:45.687444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:45.687444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:45.687444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:45.687444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:45.687444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 537/1064] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:45.689398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.752770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:45.805808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7eb9d-626e-476a-9bad-fcd804a3f2f1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:45.841601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:45.842275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:45.842553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:45.842847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:45.843626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:45.843932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:45.845314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:45.858278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 528/1065] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:45.864119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:45.971143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07f986-1fee-45ff-856d-94ec0e9d6c2e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.005204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.006002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.006284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.006551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.014284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.015759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:46.030556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:46.031584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:46.032610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:46.034849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 538/1066] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:27:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:46.038112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.142195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4322204-720d-4037-8e45-7db3bb4367e9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.160501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.180508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.181252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.181537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.181794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.182676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:46.183028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.184525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404fd0b-b166-46d5-acaf-9cf5b891d3fb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.204974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 529/1067] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:46.210889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.237683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.327811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2716-4731-4d63-85b4-fd5ac90d81fb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.363385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.364183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.364435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.364732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.376585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.378165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.382445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa9878-627f-425c-8c0a-e67b52cd966f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.382733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa9878-627f-425c-8c0a-e67b52cd966f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.383043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa9878-627f-425c-8c0a-e67b52cd966f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.384842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 530/1068] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:46.389842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.390126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.390397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:46.397596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:46.398449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:46.399414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:46.399414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:46.399414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:46.399414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:46.399414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 539/1069] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:46.407981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.413336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:46.414638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:46.488047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 531/1070] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:46.490006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.510226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.517113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270cd2b-f17b-4fce-9543-0702004a1adc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.552580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.553232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.553503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.553770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.554498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:46.554800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.556356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.568715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 540/1071] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:46.574207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.612156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c5e9-77e3-4c49-843d-10eb0e27b804 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.620008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.646989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.647842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.648066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.648335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.658719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:46.666312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:46.667317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:46.668150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:46.672973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 532/1072] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:46.673807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.676934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9431a-e411-49bb-8e7e-b9aef5706892 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.712917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.713709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.713982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.714252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.716859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.723411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.725805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:46.743749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:46.744677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:46.745552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:46.751553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 541/1073] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:46.751892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.777988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a11911-40c5-4a7f-8cd6-a6fe15d2bd97 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.822877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.823556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.823836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.824109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.825024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:46.825810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.827493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.837194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 533/1074] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:46.842852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.861015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba467ab9-2b81-458f-b41e-dade8fb7f434 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.872172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.896721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.897392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.897666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.897990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.898713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:46.899094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.900529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:46.914797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 542/1075] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:46.919052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:46.956305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c59219-ace7-4d3e-adea-0fe49ec03781 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:46.981099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:46.992026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476042-528b-422f-9338-82acb50792be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:46.992966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476042-528b-422f-9338-82acb50792be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:46.993259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476042-528b-422f-9338-82acb50792be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:46.993532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70476042-528b-422f-9338-82acb50792be None admin] PATH_INFO: `/v3/users/025ea35f75b74da3b563acfbb8cf7b49` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:46.994715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70476042-528b-422f-9338-82acb50792be None admin] RBAC: Authorizing `identity:get_user(user_id=025ea35f75b74da3b563acfbb8cf7b49)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:46.996161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70476042-528b-422f-9338-82acb50792be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:47.002584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 534/1076] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:46 2022] GET /identity/v3/users/025ea35f75b74da3b563acfbb8cf7b49 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:47.003482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476042-528b-422f-9338-82acb50792be None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:47.035152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cc76f-c380-4c97-854f-83162ec62d50 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:47.071660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:47.072557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:47.072834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:47.073104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/eeee4e4e0fc74d7db8b5cb9d0ba806a4/roles/d7930b987c3741b8ab8a3a0e213ae3d4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:47.075362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=eeee4e4e0fc74d7db8b5cb9d0ba806a4, role_id=d7930b987c3741b8ab8a3a0e213ae3d4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:47.076787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:47.092284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 543/1077] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:46 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/eeee4e4e0fc74d7db8b5cb9d0ba806a4/roles/d7930b987c3741b8ab8a3a0e213ae3d4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:27:47.104642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:47.112619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70476042-528b-422f-9338-82acb50792be None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:47.150511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccef97-c2bb-421b-ba7a-a2b2b5f62bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:47.151413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccef97-c2bb-421b-ba7a-a2b2b5f62bfa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:47.151688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccef97-c2bb-421b-ba7a-a2b2b5f62bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:47.151951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccef97-c2bb-421b-ba7a-a2b2b5f62bfa None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:47.153289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccef97-c2bb-421b-ba7a-a2b2b5f62bfa None admin] RBAC: Authorizing `identity:get_project(project_id=c6726fa6092b4f76b511ceeee08afe06)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:47.154826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccef97-c2bb-421b-ba7a-a2b2b5f62bfa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:47.156805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 535/1078] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:27:47 2022] GET /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:47.163280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:47.278116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefdd46-3010-470c-b962-aaceebad207f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:47.294093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:47.320686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc10f05-e217-41bf-b265-617bff736748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:47.321616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10f05-e217-41bf-b265-617bff736748 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:47.321895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10f05-e217-41bf-b265-617bff736748 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:47.322165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc10f05-e217-41bf-b265-617bff736748 None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06/users/025ea35f75b74da3b563acfbb8cf7b49/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:47.324304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc10f05-e217-41bf-b265-617bff736748 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6726fa6092b4f76b511ceeee08afe06, user_id=025ea35f75b74da3b563acfbb8cf7b49, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:47.325762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc10f05-e217-41bf-b265-617bff736748 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:47.342591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 544/1079] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:47 2022] PUT /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06/users/025ea35f75b74da3b563acfbb8cf7b49/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:27:47.497258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80e47-942b-4469-bd04-a5b3c133208f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:47.497455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80e47-942b-4469-bd04-a5b3c133208f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:47.497693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc80e47-942b-4469-bd04-a5b3c133208f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:47.499278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 536/1080] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:47.504895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:47.505211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:47.505395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:47.523098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:47.524452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:47.597873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 545/1081] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:47.620170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc80e47-942b-4469-bd04-a5b3c133208f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:47.738016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc80e47-942b-4469-bd04-a5b3c133208f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:47.774501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:47.775111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:47.775388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:47.775656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:47.813757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:47.853261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:47.854981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:47.955756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 537/1082] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:47.961585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:48.059745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6997f8-3fd8-4710-8fa4-bfeb98bbcc31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:48.061992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:48.096008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:48.096732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:48.097005 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:48.097269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:48.107231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:48.108672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d97a1895-6aa4-4cbb-ac1e-f8b4bc5dd401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:48.125896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:48.127104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:48.128002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:48.128002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:48.128002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:48.128002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:48.131123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 546/1083] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:48.133064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:48.199344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:48.241642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd42e5-b65d-4b7b-971d-30680adda52b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:48.276845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:48.277511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:48.277788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:48.278061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:48.278809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:48.279117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:48.280495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d8c1b-5da0-49fa-860e-91eac96a64f5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:48.299292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 538/1084] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:49.306927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab37810-4e62-40af-9efb-40b81b2de83d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.307489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab37810-4e62-40af-9efb-40b81b2de83d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.307489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab37810-4e62-40af-9efb-40b81b2de83d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.309025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 547/1085] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:49.314160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efe0f9-b026-47c6-9c3d-452497e9d479 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.314606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efe0f9-b026-47c6-9c3d-452497e9d479 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.314888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efe0f9-b026-47c6-9c3d-452497e9d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.332644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-30efe0f9-b026-47c6-9c3d-452497e9d479 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:49.333851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30efe0f9-b026-47c6-9c3d-452497e9d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:49.410795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 539/1086] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:49.434121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab37810-4e62-40af-9efb-40b81b2de83d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:49.534535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199cec8-5978-4a89-8844-065babe5e3df None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.534814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199cec8-5978-4a89-8844-065babe5e3df None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.535110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199cec8-5978-4a89-8844-065babe5e3df None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.536450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 540/1087] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:27:49.537072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab37810-4e62-40af-9efb-40b81b2de83d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:49.541669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.541956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.542228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.559847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:49.560980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:49.576922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:49.577755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.578034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.578314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.590086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:49.598235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:49.599266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:49.600101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 548/1088] 10.222.0.32 () {56 vars in 1120 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:49.605451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:49.609429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:49.623264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:49.627165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 541/1089] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:49.649920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:49.705588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78383876-5f1e-4336-88ec-7a616c908714 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:49.738025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:49.738695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.739005 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.739274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.740018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:49.740321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:49.741682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:49.748513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ad4dd-767e-43a8-8fff-51c05d34d71f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:49.749670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 549/1090] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:49.755383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:49.781766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:49.782713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.783438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.783612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.795966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:27:49.802178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:49.803284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:49.804623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 542/1091] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:49.809277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:49.838451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:49.855033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:49.860123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b8b7-2f32-4844-a47b-5f0fe234ab99 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:49.893920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:49.894714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.895013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.895276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.906165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:49.907619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:49.909287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcb2ee-6e55-4f13-ab40-913167f6d123 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:49.926284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:49.927290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:49.933010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:49.933010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:49.933010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:49.933010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:49.933010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 550/1092] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:27:49.933435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:49.946141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:49.946866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:49.947144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:49.947414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:49.948165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:49.948471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:49.949859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:49.958350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 543/1093] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:49.964055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.036950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e042f7-7839-44d6-87fe-1795bd5634fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.062313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910951ca-5812-4149-bd90-4a1bf0be94b0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.064524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.069842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.070776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.071438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.071733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.071988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.072726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:50.073030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.074371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.086047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 551/1094] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:50.091815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.099640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.100520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.100793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.101057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] PATH_INFO: `/v3/users/025ea35f75b74da3b563acfbb8cf7b49` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.102379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] RBAC: Authorizing `identity:get_user(user_id=025ea35f75b74da3b563acfbb8cf7b49)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.103861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.106495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 544/1095] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:49 2022] GET /identity/v3/users/025ea35f75b74da3b563acfbb8cf7b49 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:50.110953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.189079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0cd4-8ef7-47b2-9e06-a40e484f2daf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.207176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c654c-2555-487d-ab69-869ad07878eb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.222704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.223600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.223875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.224135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.231175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.232818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.242458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.243138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.243417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.243681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:27:50.247142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:50.248235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:27:50.249359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 552/1096] 10.222.0.32 () {56 vars in 1093 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:50.254179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.288646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.303294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.315769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.317241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.355796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c28bf2-9425-4684-8dc5-ef8fe36cbda4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.392451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.393163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.393408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.393704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.394561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:50.394891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.396328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.412977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 553/1097] 10.222.0.32 () {56 vars in 1133 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:50.419004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.427551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 545/1098] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:50.433463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.508168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.519840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e293f5-0ad8-4d3f-825c-043e1ed05a71 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.534943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.538079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32821dbf-d0fc-4889-a58f-2f22b286494e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.564399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.565061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.565338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.565610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.573823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.574692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.575000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.575274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.576536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] RBAC: Authorizing `identity:get_project(project_id=c6726fa6092b4f76b511ceeee08afe06)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.577995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.579731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 546/1099] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:50.585709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.634270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.636056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.689884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ee00-a146-479c-a533-1c3c38f214f6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.724092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.724744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.725024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.725300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.726142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=c6726fa6092b4f76b511ceeee08afe06, user.id=025ea35f75b74da3b563acfbb8cf7b49) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:50.726456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.727966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.733819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 547/1100] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/role_assignments?user.id=025ea35f75b74da3b563acfbb8cf7b49&scope.project.id=c6726fa6092b4f76b511ceeee08afe06&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:50.734342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 554/1101] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 14:27:50.740375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.750811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:50.850940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9e18b-b5e0-45a1-9204-44a997595f7f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:50.887585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:50.888305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:50.888578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:50.888845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:50.900919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:50.902405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:27:50.921220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:50.922570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:50.923490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:27:50.923490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:27:50.923490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:27:50.923490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:50.923490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 548/1102] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:50.928959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:50.939776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:51.032607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a7cd2-edbb-40d4-b8a0-cd3f8eea6b98 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:51.067729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:51.068391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:51.068663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:51.068930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:51.069650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:51.069948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:51.071462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:51.083099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 555/1103] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:51.088661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:51.152128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:51.195872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c035e25e-d600-4f24-8b41-069558db403f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:51.230348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:51.231203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:51.231481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:51.231753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:51.239226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:51.241290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:27:51.255738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:27:51.256655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:27:51.257685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 549/1104] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:27:51.263072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:51.332490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:51.368575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e403da-0ebc-408d-9537-bc1facd01348 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:51.402828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:51.403459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:51.403742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:51.404014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:51.404805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:51.405107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:51.406541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c8bbb-d5b4-485e-9277-5dfa03062136 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:51.418774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 556/1105] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:51.425481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:51.531217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e411d-48f7-4840-95e9-ab4ee808a5e1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:51.550104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:51.568010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:51.568668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:51.568941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:51.569207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:51.570039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] RBAC: Adding query filter params (role.id=d7930b987c3741b8ab8a3a0e213ae3d4, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:51.570347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:51.571937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75f863-5592-47dd-9b07-8128974642aa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:51.577848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 550/1106] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:51 2022] GET /identity/v3/role_assignments?user.id=eeee4e4e0fc74d7db8b5cb9d0ba806a4&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=d7930b987c3741b8ab8a3a0e213ae3d4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:52.934221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577d12-132c-4ad7-85de-c53710db6912 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:52.935120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577d12-132c-4ad7-85de-c53710db6912 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:52.935120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577d12-132c-4ad7-85de-c53710db6912 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:52.937385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 557/1107] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:52.943716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:52.944151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:52.944276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:52.970221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:52.971443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:53.042227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 551/1108] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:53.067795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577d12-132c-4ad7-85de-c53710db6912 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:53.171143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577d12-132c-4ad7-85de-c53710db6912 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:53.198254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:53.208204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be31ca-d8b8-4402-aa6c-c1315d455957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:53.208992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be31ca-d8b8-4402-aa6c-c1315d455957 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:53.209266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be31ca-d8b8-4402-aa6c-c1315d455957 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:53.209537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be31ca-d8b8-4402-aa6c-c1315d455957 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:53.210849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be31ca-d8b8-4402-aa6c-c1315d455957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:53.212293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be31ca-d8b8-4402-aa6c-c1315d455957 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:53.213911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 558/1109] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:27:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:53.223083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:53.328447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9537f7e-99f2-4a9d-8e94-99b26fc09b2a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:53.341305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:53.363979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f7929-36a9-47f8-829a-7b666f7a5971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:53.364642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7929-36a9-47f8-829a-7b666f7a5971 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:53.364920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7929-36a9-47f8-829a-7b666f7a5971 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:53.365192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f7929-36a9-47f8-829a-7b666f7a5971 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:53.365895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7929-36a9-47f8-829a-7b666f7a5971 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:53.367397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f7929-36a9-47f8-829a-7b666f7a5971 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:53.395022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 552/1110] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:27:53 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:27:55.619870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9563d3-2eea-4ff0-8af6-b1143177eaa9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:55.620475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9563d3-2eea-4ff0-8af6-b1143177eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:55.620475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9563d3-2eea-4ff0-8af6-b1143177eaa9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:55.621815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 559/1111] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:55.628430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:55.629197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:55.629326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:55.659367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:55.661266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:55.734373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 553/1112] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:55.757905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9563d3-2eea-4ff0-8af6-b1143177eaa9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:55.879309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9563d3-2eea-4ff0-8af6-b1143177eaa9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:55.914539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:55.915418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:55.915696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:55.915963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] PATH_INFO: `/v3/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:55.917552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] RBAC: Authorizing `identity:get_role(role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:55.918953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:55.920328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 560/1113] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:55 2022] GET /identity/v3/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:55.927503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:56.024971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:56.029875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679c9d0-f03b-4ee1-b4e0-e8b557643139 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:56.074294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:56.075199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:56.075478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:56.075751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] PATH_INFO: `/v3/users/301289c1cf82440082fb766ba3aef3c6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:56.088627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] RBAC: Authorizing `identity:get_user(user_id=301289c1cf82440082fb766ba3aef3c6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:56.090083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:56.091779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 554/1114] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:55 2022] GET /identity/v3/users/301289c1cf82440082fb766ba3aef3c6 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:56.098458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:56.163841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:56.216683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ede1-0e25-454e-bfc4-2718648151b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:56.252434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:56.253044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:56.253320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:56.253593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:56.330263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:56.331735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:56.394210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:56.439978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 561/1115] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:27:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:27:56.446108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:56.547749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6158eb-fa6e-4f2a-8112-4e4dbcd3afd6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:56.582578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902d1ce-e4f8-49d5-b5a7-ec34d5e3c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:56.583451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902d1ce-e4f8-49d5-b5a7-ec34d5e3c0ce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:56.583725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902d1ce-e4f8-49d5-b5a7-ec34d5e3c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:56.583994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902d1ce-e4f8-49d5-b5a7-ec34d5e3c0ce None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:56.585194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902d1ce-e4f8-49d5-b5a7-ec34d5e3c0ce None admin] RBAC: Authorizing `identity:get_project(project_id=c6726fa6092b4f76b511ceeee08afe06)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:56.586686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902d1ce-e4f8-49d5-b5a7-ec34d5e3c0ce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:56.588376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 555/1116] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:27:56 2022] GET /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:56.595328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:56.709129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77cac3-03d1-4b4c-9fe5-9de98df799c3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:56.747896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:56.748540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:56.748815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:56.749082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:56.749909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] RBAC: Adding query filter params (role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=c6726fa6092b4f76b511ceeee08afe06, user.id=301289c1cf82440082fb766ba3aef3c6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:27:56.750216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:56.751656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9c8cf-1c48-4c36-86a7-b3a196256b46 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:56.757436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 562/1117] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:27:56 2022] GET /identity/v3/role_assignments?user.id=301289c1cf82440082fb766ba3aef3c6&scope.project.id=c6726fa6092b4f76b511ceeee08afe06&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:58.867507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85f41b-9eec-4280-9eaf-9f954ee71560 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:58.868032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85f41b-9eec-4280-9eaf-9f954ee71560 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:58.868032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85f41b-9eec-4280-9eaf-9f954ee71560 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:58.869392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 556/1118] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:27:58.874850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:58.875271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:58.875383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:58.893855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:27:58.895051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:27:58.918326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:58.966014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 563/1119] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:27:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:27:58.988319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85f41b-9eec-4280-9eaf-9f954ee71560 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:59.017320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:59.096763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85f41b-9eec-4280-9eaf-9f954ee71560 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:59.131210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:59.132064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:59.132336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:59.132755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] PATH_INFO: `/v3/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:59.134269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] RBAC: Authorizing `identity:get_role(role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:59.135686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:59.137058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 557/1120] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:58 2022] GET /identity/v3/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:59.145849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:59.247782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f258a9-8b5c-48c3-8317-dd5aa05a339b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:59.282401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:59.283286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:59.283564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:59.283830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] PATH_INFO: `/v3/users/301289c1cf82440082fb766ba3aef3c6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:59.285114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] RBAC: Authorizing `identity:get_user(user_id=301289c1cf82440082fb766ba3aef3c6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:59.286539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:59.288164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 564/1121] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:27:59 2022] GET /identity/v3/users/301289c1cf82440082fb766ba3aef3c6 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:27:59.297021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:59.391643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:59.402611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afb614-1851-4ad3-adc4-6e638ebb3d37 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:59.441936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652472e8-c24c-42fb-a406-c7078bc940d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:59.442797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652472e8-c24c-42fb-a406-c7078bc940d0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:59.443092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652472e8-c24c-42fb-a406-c7078bc940d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:59.443362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652472e8-c24c-42fb-a406-c7078bc940d0 None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:59.444674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652472e8-c24c-42fb-a406-c7078bc940d0 None admin] RBAC: Authorizing `identity:get_project(project_id=c6726fa6092b4f76b511ceeee08afe06)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:59.446158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652472e8-c24c-42fb-a406-c7078bc940d0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:59.447870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 558/1122] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:27:59 2022] GET /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:27:59.457390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:27:59.487027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:27:59.563960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600934e-9275-4b9a-b467-4964351b1b93 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:27:59.609646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40a51b-d4a0-44a7-978d-6338abc282b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:27:59.610533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40a51b-d4a0-44a7-978d-6338abc282b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:27:59.610952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40a51b-d4a0-44a7-978d-6338abc282b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:27:59.611251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40a51b-d4a0-44a7-978d-6338abc282b9 None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06/users/301289c1cf82440082fb766ba3aef3c6/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:27:59.613857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40a51b-d4a0-44a7-978d-6338abc282b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6726fa6092b4f76b511ceeee08afe06, user_id=301289c1cf82440082fb766ba3aef3c6, role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:27:59.615339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40a51b-d4a0-44a7-978d-6338abc282b9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:27:59.631867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 565/1123] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:27:59 2022] PUT /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06/users/301289c1cf82440082fb766ba3aef3c6/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:28:01.776264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0280df-72b9-4eda-b07e-9182272b662b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:01.776870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0280df-72b9-4eda-b07e-9182272b662b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:01.776870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0280df-72b9-4eda-b07e-9182272b662b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:01.778179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 559/1124] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:01.784853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:01.785580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:01.785839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:01.810693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:01.811917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:01.880859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 566/1125] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:01.912078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0280df-72b9-4eda-b07e-9182272b662b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:02.024659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0280df-72b9-4eda-b07e-9182272b662b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:02.061272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:02.062046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:02.062320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:02.062588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] PATH_INFO: `/v3/roles/ad141ac7a7704197b36c16d7ac714c85` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:02.064139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] RBAC: Authorizing `identity:get_role(role_id=ad141ac7a7704197b36c16d7ac714c85)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:02.065507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:02.066881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 560/1126] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:01 2022] GET /identity/v3/roles/ad141ac7a7704197b36c16d7ac714c85 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:02.073674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:02.101461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:02.190651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e13cae-a55c-42cf-967f-8f819baa7d81 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:02.225730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:02.226623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:02.226923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:02.227194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] PATH_INFO: `/v3/users/301289c1cf82440082fb766ba3aef3c6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:02.228390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] RBAC: Authorizing `identity:get_user(user_id=301289c1cf82440082fb766ba3aef3c6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:02.229817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:02.231489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 567/1127] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:02 2022] GET /identity/v3/users/301289c1cf82440082fb766ba3aef3c6 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:02.237602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:02.268491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:02.344392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b75b7-f191-4a8b-aefa-c15a8f164a55 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:02.386918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:02.387566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:02.387840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:02.388105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:02.461601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:02.463088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:02.490171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:02.563638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 561/1128] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:28:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:28:02.570428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:02.666540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:02.689183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611aab0c-6f33-4366-9c7c-423a5d053ccc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:02.730146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cac3df-f586-43fb-a2c5-5ba085306139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:02.731034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cac3df-f586-43fb-a2c5-5ba085306139 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:02.731315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cac3df-f586-43fb-a2c5-5ba085306139 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:02.731580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cac3df-f586-43fb-a2c5-5ba085306139 None admin] PATH_INFO: `/v3/projects/c6726fa6092b4f76b511ceeee08afe06` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:02.733431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cac3df-f586-43fb-a2c5-5ba085306139 None admin] RBAC: Authorizing `identity:get_project(project_id=c6726fa6092b4f76b511ceeee08afe06)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:02.735513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cac3df-f586-43fb-a2c5-5ba085306139 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:02.737826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 568/1129] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:02 2022] GET /identity/v3/projects/c6726fa6092b4f76b511ceeee08afe06 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:02.744468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:02.854679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ab5a5-f10b-42fa-94ff-90ee8f3cc747 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:02.882149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:02.891770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:02.892494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:02.892770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:02.893039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:02.893857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] RBAC: Adding query filter params (role.id=ad141ac7a7704197b36c16d7ac714c85, scope.project.id=c6726fa6092b4f76b511ceeee08afe06, user.id=301289c1cf82440082fb766ba3aef3c6) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:02.894167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:02.895637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f4a1-984d-4f5a-a94f-24720871d6ec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:02.901807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 562/1130] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:28:02 2022] GET /identity/v3/role_assignments?user.id=301289c1cf82440082fb766ba3aef3c6&scope.project.id=c6726fa6092b4f76b511ceeee08afe06&role.id=ad141ac7a7704197b36c16d7ac714c85 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:05.092661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f58f8-93d0-4f0c-9dba-39887e62a5b1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:05.093497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f58f8-93d0-4f0c-9dba-39887e62a5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:05.093497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f58f8-93d0-4f0c-9dba-39887e62a5b1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:05.095810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 569/1131] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:05.101275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:05.101697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:05.101786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:05.120258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:05.121429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:05.189499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 563/1132] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:05.213673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f58f8-93d0-4f0c-9dba-39887e62a5b1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:05.334006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f58f8-93d0-4f0c-9dba-39887e62a5b1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:05.369950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270e717-9de5-4a0c-b9fd-8e26855c6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:05.371036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e717-9de5-4a0c-b9fd-8e26855c6e99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:05.371413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e717-9de5-4a0c-b9fd-8e26855c6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:05.371771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e717-9de5-4a0c-b9fd-8e26855c6e99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:05.373759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e717-9de5-4a0c-b9fd-8e26855c6e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:05.376058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e717-9de5-4a0c-b9fd-8e26855c6e99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:05.378588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 570/1133] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:28:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:05.386959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:05.491843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0b31-3472-4b17-a90e-839a59e4d7a3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:05.526669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:05.539966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a30fcd-df63-4f26-9a71-d6ca19f73a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:05.540610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30fcd-df63-4f26-9a71-d6ca19f73a58 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:05.540885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30fcd-df63-4f26-9a71-d6ca19f73a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:05.541155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a30fcd-df63-4f26-9a71-d6ca19f73a58 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:05.541845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30fcd-df63-4f26-9a71-d6ca19f73a58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:05.543330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a30fcd-df63-4f26-9a71-d6ca19f73a58 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:05.572473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 564/1134] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:28:05 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:28:07.777273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbfa88-f47d-4ae6-a9fb-bc0a7b6c8add None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:07.778136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbfa88-f47d-4ae6-a9fb-bc0a7b6c8add None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:07.778136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbfa88-f47d-4ae6-a9fb-bc0a7b6c8add None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:07.780315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 571/1135] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:07.789346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:07.789792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:07.789890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:07.812140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:07.813332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:07.883717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 565/1136] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:07.908159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbfa88-f47d-4ae6-a9fb-bc0a7b6c8add None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:07.933024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:08.017098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbfa88-f47d-4ae6-a9fb-bc0a7b6c8add None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:08.061748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d715a6-e752-4a03-9106-14c0aad2d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:08.062831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d715a6-e752-4a03-9106-14c0aad2d929 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:08.063221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d715a6-e752-4a03-9106-14c0aad2d929 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:08.063573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d715a6-e752-4a03-9106-14c0aad2d929 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:08.065650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d715a6-e752-4a03-9106-14c0aad2d929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:08.068007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d715a6-e752-4a03-9106-14c0aad2d929 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:08.070326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 572/1137] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:28:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:08.078656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:08.182198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b1558-1485-41bb-bfe4-b7e74d848584 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:08.184559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:08.216682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72802f2b-f3c3-4783-a5ee-bebdd076a19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:08.217350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72802f2b-f3c3-4783-a5ee-bebdd076a19e None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:08.217625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72802f2b-f3c3-4783-a5ee-bebdd076a19e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:08.217895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72802f2b-f3c3-4783-a5ee-bebdd076a19e None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:08.218593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72802f2b-f3c3-4783-a5ee-bebdd076a19e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:08.220080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72802f2b-f3c3-4783-a5ee-bebdd076a19e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:08.242018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 566/1138] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:28:08 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:28:10.461192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e201d-cc3c-422b-a32c-6928eba4ab5a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:10.461925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e201d-cc3c-422b-a32c-6928eba4ab5a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:10.461925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e201d-cc3c-422b-a32c-6928eba4ab5a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:10.463810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 573/1139] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:10.472958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:10.473397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:10.473486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:10.493419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:10.494609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:10.566199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 567/1140] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:10.590070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e201d-cc3c-422b-a32c-6928eba4ab5a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:10.691341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:10.706482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e201d-cc3c-422b-a32c-6928eba4ab5a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:10.742318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:10.743077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:10.743355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:10.743621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:10.754211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:28:10.761913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:10.762968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:28:10.763936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 574/1141] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:28:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:10.769834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:10.865498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:10.882260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7f8e1-d587-43df-97a3-d2a9d2603728 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:10.924513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:10.925304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:10.925437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:10.925727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:10.926484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:10.926799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:10.928217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:10.936566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 568/1142] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:28:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:10.942679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:11.045995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a36463-5d48-4889-94d0-69b1be7289d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:11.080362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:11.081227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:11.081509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:11.081779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] PATH_INFO: `/v3/users/f9a3cbf1a9fb4cf0804ce941ca2a1618` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:11.095381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] RBAC: Authorizing `identity:get_user(user_id=f9a3cbf1a9fb4cf0804ce941ca2a1618)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:11.096822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:11.098457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 575/1143] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:10 2022] GET /identity/v3/users/f9a3cbf1a9fb4cf0804ce941ca2a1618 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:11.104363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:11.216101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d81d8e-e722-408c-8b7e-93224d6f0808 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:11.255028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:11.255681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:11.255958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:11.256236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:11.279044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:11.327401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:11.328796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:11.434984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 569/1144] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:28:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:28:11.441751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:11.452426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:11.546468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580a82d-c4fa-4021-89c4-8cd07673ca7b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:11.581691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d650b8-37ed-434f-ae75-53ce7afe775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:11.582463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d650b8-37ed-434f-ae75-53ce7afe775b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:11.582741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d650b8-37ed-434f-ae75-53ce7afe775b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:11.583034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d650b8-37ed-434f-ae75-53ce7afe775b None admin] PATH_INFO: `/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:11.584318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d650b8-37ed-434f-ae75-53ce7afe775b None admin] RBAC: Authorizing `identity:get_project(project_id=a0d90f4cd4a64abe8ad9fda6a89f88cf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:11.585826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d650b8-37ed-434f-ae75-53ce7afe775b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:11.587531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 576/1145] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:11 2022] GET /identity/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:11.597301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:11.667442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:11.716599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57fafe-0597-4e5d-929c-8215c35fd303 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:11.756707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:11.757685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:11.758064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:11.758423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:11.759788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=a0d90f4cd4a64abe8ad9fda6a89f88cf, user.id=f9a3cbf1a9fb4cf0804ce941ca2a1618) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:11.760208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:11.762446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0a10-b51c-47a0-ba8f-d57840f96e00 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:11.771455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 570/1146] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:28:11 2022] GET /identity/v3/role_assignments?user.id=f9a3cbf1a9fb4cf0804ce941ca2a1618&scope.project.id=a0d90f4cd4a64abe8ad9fda6a89f88cf&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:13.880852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bba51-9752-4923-9fe0-e22ec0372e69 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:13.881358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bba51-9752-4923-9fe0-e22ec0372e69 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:13.881358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bba51-9752-4923-9fe0-e22ec0372e69 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:13.882784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 577/1147] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:13.890290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:13.890808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:13.891722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:13.910715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:13.911932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:13.983714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 571/1148] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:14.009812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bba51-9752-4923-9fe0-e22ec0372e69 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:14.098113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:14.125016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bba51-9752-4923-9fe0-e22ec0372e69 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:14.163907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:14.164686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:14.164958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:14.165230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:14.175810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:28:14.183444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:14.184475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:28:14.185416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 578/1149] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:28:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:14.193290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:14.239580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:14.293232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f209f7-1237-44bc-b950-2677869fe8c9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:14.330584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:14.331279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:14.331556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:14.331823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:14.332566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:14.332868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:14.334246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:14.343150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 572/1150] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:28:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:28:14.352889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:14.471527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb416191-1612-4863-93e4-82bab7bfb770 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:14.510558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:14.511466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:14.511739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:14.512009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] PATH_INFO: `/v3/users/f9a3cbf1a9fb4cf0804ce941ca2a1618` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:14.513242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] RBAC: Authorizing `identity:get_user(user_id=f9a3cbf1a9fb4cf0804ce941ca2a1618)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:14.514686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:14.516365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 579/1151] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:14 2022] GET /identity/v3/users/f9a3cbf1a9fb4cf0804ce941ca2a1618 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:14.522655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:14.636493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83d526-cc37-401e-9972-39eabf783e2a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:14.646285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:14.673972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb28dd3-80d7-4ecf-8ca3-6782b84ed602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:14.674855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28dd3-80d7-4ecf-8ca3-6782b84ed602 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:14.675137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28dd3-80d7-4ecf-8ca3-6782b84ed602 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:14.675410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb28dd3-80d7-4ecf-8ca3-6782b84ed602 None admin] PATH_INFO: `/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:14.676714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb28dd3-80d7-4ecf-8ca3-6782b84ed602 None admin] RBAC: Authorizing `identity:get_project(project_id=a0d90f4cd4a64abe8ad9fda6a89f88cf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:14.678167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb28dd3-80d7-4ecf-8ca3-6782b84ed602 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:14.680051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 573/1152] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:14 2022] GET /identity/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:14.686198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:14.689253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:14.794120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6f12-55bc-419c-acad-9e441fa817a3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:14.833079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b45884-8097-4b9c-a937-361d39b70f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:14.834033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b45884-8097-4b9c-a937-361d39b70f26 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:14.834314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b45884-8097-4b9c-a937-361d39b70f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:14.834579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b45884-8097-4b9c-a937-361d39b70f26 None admin] PATH_INFO: `/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf/users/f9a3cbf1a9fb4cf0804ce941ca2a1618/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:14.836816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b45884-8097-4b9c-a937-361d39b70f26 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0d90f4cd4a64abe8ad9fda6a89f88cf, user_id=f9a3cbf1a9fb4cf0804ce941ca2a1618, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:14.838274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b45884-8097-4b9c-a937-361d39b70f26 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:14.855595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 580/1153] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:28:14 2022] PUT /identity/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf/users/f9a3cbf1a9fb4cf0804ce941ca2a1618/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:28:16.982513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e211e8-53f0-4fb0-8f0a-e6cdb772b1f7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:16.983554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e211e8-53f0-4fb0-8f0a-e6cdb772b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:16.983554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e211e8-53f0-4fb0-8f0a-e6cdb772b1f7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:16.985602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 574/1154] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:16.992206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:16.992659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:16.992763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:17.020324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:17.022209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:17.083699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:17.096226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 581/1155] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:17.121007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e211e8-53f0-4fb0-8f0a-e6cdb772b1f7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:17.228865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e211e8-53f0-4fb0-8f0a-e6cdb772b1f7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:17.265481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:17.266276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:17.266527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:17.266795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:17.277630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:28:17.285148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:17.286162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:28:17.287094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 575/1156] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:28:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:17.293257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:17.417372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff0ce8-a35f-4bb8-9aea-a0efbda9e8fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:17.455172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:17.455840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:17.456118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:17.456386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:17.457137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:17.457435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:17.458855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:17.467644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 582/1157] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:28:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:17.473774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:17.516110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:17.585274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd140d-64ad-4e6d-a8fa-738330fd799f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:17.622188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:17.623138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:17.623413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:17.623715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] PATH_INFO: `/v3/users/f9a3cbf1a9fb4cf0804ce941ca2a1618` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:17.625021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] RBAC: Authorizing `identity:get_user(user_id=f9a3cbf1a9fb4cf0804ce941ca2a1618)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:17.626493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:17.629153 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 576/1158] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:17 2022] GET /identity/v3/users/f9a3cbf1a9fb4cf0804ce941ca2a1618 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:17.635562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:17.697275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:17.743959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26999731-88dd-44d8-8db7-30fa5ee36782 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:17.778651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:17.779340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:17.779617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:17.779888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:17.852713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:17.854163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:17.912932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:17.962167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 583/1159] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:28:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:28:17.968822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:18.074270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ab04-0057-456d-b90d-415c54a421d2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:18.076644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:18.119786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98ea68-6aed-4f4c-8606-5b1498b030fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:18.120653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98ea68-6aed-4f4c-8606-5b1498b030fb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:18.120929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98ea68-6aed-4f4c-8606-5b1498b030fb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:18.121194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98ea68-6aed-4f4c-8606-5b1498b030fb None admin] PATH_INFO: `/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:18.122460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98ea68-6aed-4f4c-8606-5b1498b030fb None admin] RBAC: Authorizing `identity:get_project(project_id=a0d90f4cd4a64abe8ad9fda6a89f88cf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:18.123966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98ea68-6aed-4f4c-8606-5b1498b030fb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:18.125651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 577/1160] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:17 2022] GET /identity/v3/projects/a0d90f4cd4a64abe8ad9fda6a89f88cf => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:18.132161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:18.235763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de676e-8f35-4500-9c66-d2f0a7c0f81e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:18.274069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:18.274722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:18.275034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:18.275305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:18.276139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=a0d90f4cd4a64abe8ad9fda6a89f88cf, user.id=f9a3cbf1a9fb4cf0804ce941ca2a1618) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:18.276451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:18.277882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a4ac1-2eea-4331-880a-b9469a67333d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:18.284004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 584/1161] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:28:18 2022] GET /identity/v3/role_assignments?user.id=f9a3cbf1a9fb4cf0804ce941ca2a1618&scope.project.id=a0d90f4cd4a64abe8ad9fda6a89f88cf&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:20.463254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878358d0-f962-4c37-8282-4e678789de8b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:20.464172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878358d0-f962-4c37-8282-4e678789de8b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:20.464172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878358d0-f962-4c37-8282-4e678789de8b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:20.466258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 578/1162] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:20.472608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:20.473056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:20.473395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:20.498233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:20.499547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:20.513007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:20.579359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 585/1163] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:20.602071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878358d0-f962-4c37-8282-4e678789de8b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:20.715457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878358d0-f962-4c37-8282-4e678789de8b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:20.751714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:20.752359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:20.752638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:20.752905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:20.801053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:20.828558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:20.830019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:20.936518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 579/1164] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:28:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:28:20.942941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:21.046613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00bd15-9366-4a89-80f9-5db2e6fe423e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:21.081855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:21.082649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:21.082953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:21.083221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:21.090306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:21.091776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-34d9da59-a9db-455c-bf33-38535f9260b1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:21.106284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:21.107481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:28:21.108485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 586/1165] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:28:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:21.114191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:21.193975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:21.229294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4678c-c1eb-4f8b-86c6-d0b0d98a6f28 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:21.264252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:21.264903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:21.265177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:21.265438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:21.266154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:21.266471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:21.267857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fec633-3df9-4d77-a576-0ba7a8ab56d0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:21.275112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 580/1166] 10.222.0.32 () {56 vars in 1115 bytes} [Wed May 25 14:28:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:28:23.457256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d92c-9231-448c-9041-4d1a6f0ca686 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:23.458082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d92c-9231-448c-9041-4d1a6f0ca686 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:23.458082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238d92c-9231-448c-9041-4d1a6f0ca686 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:23.460058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 587/1167] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:28:23.465528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915fdd-01f2-4835-bcea-e603c1fad3a5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:23.465987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915fdd-01f2-4835-bcea-e603c1fad3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:23.466078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915fdd-01f2-4835-bcea-e603c1fad3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:23.484079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d2915fdd-01f2-4835-bcea-e603c1fad3a5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:23.485276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d2915fdd-01f2-4835-bcea-e603c1fad3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:23.554341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 581/1168] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:23.589251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d92c-9231-448c-9041-4d1a6f0ca686 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:23.608453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:23.711973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238d92c-9231-448c-9041-4d1a6f0ca686 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:23.747262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f791d8d-d3fc-4664-8514-1a695d2d7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:23.747901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f791d8d-d3fc-4664-8514-1a695d2d7d67 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:23.748171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f791d8d-d3fc-4664-8514-1a695d2d7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:23.748438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f791d8d-d3fc-4664-8514-1a695d2d7d67 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:23.749164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f791d8d-d3fc-4664-8514-1a695d2d7d67 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:23.750518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f791d8d-d3fc-4664-8514-1a695d2d7d67 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:23.772793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 588/1169] 10.222.0.32 () {60 vars in 1138 bytes} [Wed May 25 14:28:23 2022] POST /identity/v3/domains => generated 267 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:28:25.948055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27645de9-796a-468d-be9d-33860311df3d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:25.948966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27645de9-796a-468d-be9d-33860311df3d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:25.948966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27645de9-796a-468d-be9d-33860311df3d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:25.951213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 582/1170] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:25.956820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:25.957241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:25.957354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:25.975758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:25.976921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:26.048721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:26.054829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 589/1171] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:26.078314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27645de9-796a-468d-be9d-33860311df3d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:26.146336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:26.194002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27645de9-796a-468d-be9d-33860311df3d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:26.241479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c1cb5-a1d2-4d0c-8dc1-6dbbd7a59a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:26.242331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c1cb5-a1d2-4d0c-8dc1-6dbbd7a59a5d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:26.242608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c1cb5-a1d2-4d0c-8dc1-6dbbd7a59a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:26.242914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c1cb5-a1d2-4d0c-8dc1-6dbbd7a59a5d None admin] PATH_INFO: `/v3/domains/f90ba128b3a246189e62dbb469b4107b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:26.253885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c1cb5-a1d2-4d0c-8dc1-6dbbd7a59a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=f90ba128b3a246189e62dbb469b4107b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:26.255379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c1cb5-a1d2-4d0c-8dc1-6dbbd7a59a5d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:26.257069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 583/1172] 10.222.0.32 () {56 vars in 1183 bytes} [Wed May 25 14:28:26 2022] GET /identity/v3/domains/f90ba128b3a246189e62dbb469b4107b => generated 267 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:26.265844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:26.369831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff424d3-b1d2-471d-a595-3b327761a8ed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:26.403915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093265-2fe9-40de-86a2-4cff2d5044ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:26.404560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093265-2fe9-40de-86a2-4cff2d5044ab None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:26.404839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093265-2fe9-40de-86a2-4cff2d5044ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:26.405110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093265-2fe9-40de-86a2-4cff2d5044ab None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:26.405797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093265-2fe9-40de-86a2-4cff2d5044ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:26.407253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093265-2fe9-40de-86a2-4cff2d5044ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:26.436544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 590/1173] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:28:26 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:28:28.633375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce997c-cb53-4eff-a027-f0daba223eac None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:28.633948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce997c-cb53-4eff-a027-f0daba223eac None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:28.633948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce997c-cb53-4eff-a027-f0daba223eac None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:28.636064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 584/1174] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:28.643555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:28.644020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:28.644112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:28.663370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:28.664576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:28.714230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:28.745855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 591/1175] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:28.771201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce997c-cb53-4eff-a027-f0daba223eac None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:28.885136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce997c-cb53-4eff-a027-f0daba223eac None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:28.910184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:28.923154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65874046-b0ed-4697-8c6e-7499e99376c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:28.924422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65874046-b0ed-4697-8c6e-7499e99376c2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:28.924845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65874046-b0ed-4697-8c6e-7499e99376c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:28.925172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65874046-b0ed-4697-8c6e-7499e99376c2 None admin] PATH_INFO: `/v3/domains/f90ba128b3a246189e62dbb469b4107b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:28.927208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65874046-b0ed-4697-8c6e-7499e99376c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f90ba128b3a246189e62dbb469b4107b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:28.928969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65874046-b0ed-4697-8c6e-7499e99376c2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:28.930594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 585/1176] 10.222.0.32 () {56 vars in 1183 bytes} [Wed May 25 14:28:28 2022] GET /identity/v3/domains/f90ba128b3a246189e62dbb469b4107b => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:28.936562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:29.049146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d02b4-423b-4419-9e4d-5781d9f87cc1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:29.083467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee7ca-0783-4aea-8cab-e5f821b97a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:29.084164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee7ca-0783-4aea-8cab-e5f821b97a8a None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:29.084438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee7ca-0783-4aea-8cab-e5f821b97a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:29.084702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee7ca-0783-4aea-8cab-e5f821b97a8a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:29.085403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee7ca-0783-4aea-8cab-e5f821b97a8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:29.086856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee7ca-0783-4aea-8cab-e5f821b97a8a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:29.109132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 592/1177] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:28:28 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:28:31.336892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403b24-b84e-4249-9d2b-799d14cc445a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:31.337768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403b24-b84e-4249-9d2b-799d14cc445a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:31.337768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403b24-b84e-4249-9d2b-799d14cc445a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:31.340063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 586/1178] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:31.346387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:31.346846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:31.346954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:31.366443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:31.367678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:31.401834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:31.442897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 593/1179] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:31.467022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403b24-b84e-4249-9d2b-799d14cc445a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:31.605178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403b24-b84e-4249-9d2b-799d14cc445a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:31.641913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:31.642704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:31.643020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:31.643294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:31.653902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:28:31.661607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:31.662679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:28:31.663550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 587/1180] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:31.669305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:31.784849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195c491-18d7-428b-983d-cb8a90f0bd47 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:31.822101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:31.822804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:31.823070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:31.823346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:31.824088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:31.824391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:31.825754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:31.834217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 594/1181] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:31.841455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:31.908183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:31.957794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f911442-52e3-4974-bf6a-78e2b9d1ca1a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:31.993233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:31.994102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:31.994382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:31.994651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] PATH_INFO: `/v3/users/b19710ac269845e5af1754dc661f26b8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:32.008362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] RBAC: Authorizing `identity:get_user(user_id=b19710ac269845e5af1754dc661f26b8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:32.009821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:32.011480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 588/1182] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:31 2022] GET /identity/v3/users/b19710ac269845e5af1754dc661f26b8 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:32.020781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:32.051226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:32.128804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f097e-bc59-4034-ae2a-6da948feb044 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:32.163210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:32.163783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:32.164106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:32.164377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:32.237269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:32.238699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:32.265555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:32.341404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 595/1183] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:28:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:28:32.348358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:32.457960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236d596-cd2c-43f1-94ab-1c743fc6bbaa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:32.470981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:32.493757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71adcc-bef5-4d10-b68e-5b244fe00d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:32.494613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71adcc-bef5-4d10-b68e-5b244fe00d82 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:32.494915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71adcc-bef5-4d10-b68e-5b244fe00d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:32.495183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71adcc-bef5-4d10-b68e-5b244fe00d82 None admin] PATH_INFO: `/v3/projects/248bba28e2444036868485767be8fddb` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:32.496465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71adcc-bef5-4d10-b68e-5b244fe00d82 None admin] RBAC: Authorizing `identity:get_project(project_id=248bba28e2444036868485767be8fddb)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:32.497905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71adcc-bef5-4d10-b68e-5b244fe00d82 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:32.499625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 589/1184] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:32 2022] GET /identity/v3/projects/248bba28e2444036868485767be8fddb => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:32.509553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:32.622733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528173-088b-4526-9ead-c53d42944c88 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:32.655431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:32.668305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:32.669638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:32.670193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:32.670650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:32.672075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=248bba28e2444036868485767be8fddb, user.id=b19710ac269845e5af1754dc661f26b8) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:32.672592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:32.674946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8796-8751-49dc-b46a-011871dd4a73 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:32.682641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 596/1185] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:28:32 2022] GET /identity/v3/role_assignments?user.id=b19710ac269845e5af1754dc661f26b8&scope.project.id=248bba28e2444036868485767be8fddb&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:34.855752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac30ba-67a9-459f-8b1b-fb9e24a9b1ee None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:34.856270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac30ba-67a9-459f-8b1b-fb9e24a9b1ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:34.856361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac30ba-67a9-459f-8b1b-fb9e24a9b1ee None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:34.858423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 590/1186] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:34.867518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:34.867935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:34.868053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:34.887284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:34.888477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:34.957220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 597/1187] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:34.979666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac30ba-67a9-459f-8b1b-fb9e24a9b1ee None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:35.094650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac30ba-67a9-459f-8b1b-fb9e24a9b1ee None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:35.129647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:35.130444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:35.130721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:35.131025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:35.141732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:28:35.149406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:35.150463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:28:35.151349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 591/1188] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:28:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:35.159327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:35.265010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ac918-6a00-4b7a-92ff-84ba78cf8995 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:35.292411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:35.306513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:35.307280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:35.307559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:35.307828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:35.308563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:35.308866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:35.310228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:35.319209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 598/1189] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:28:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:35.328208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:35.371036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:35.433049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de29faa-81e2-4cbd-87a4-69505ad1af65 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:35.468207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:35.469068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:35.469346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:35.469614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] PATH_INFO: `/v3/users/b19710ac269845e5af1754dc661f26b8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:35.470948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] RBAC: Authorizing `identity:get_user(user_id=b19710ac269845e5af1754dc661f26b8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:35.472395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:35.473997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 592/1190] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:35 2022] GET /identity/v3/users/b19710ac269845e5af1754dc661f26b8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:35.483477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:35.586339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420d72-5506-41c3-9a49-1f37936cffec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:35.620891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501a0da-964d-476a-9740-03262564d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:35.621805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501a0da-964d-476a-9740-03262564d967 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:35.621984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501a0da-964d-476a-9740-03262564d967 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:35.622273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501a0da-964d-476a-9740-03262564d967 None admin] PATH_INFO: `/v3/projects/248bba28e2444036868485767be8fddb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:35.623579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501a0da-964d-476a-9740-03262564d967 None admin] RBAC: Authorizing `identity:get_project(project_id=248bba28e2444036868485767be8fddb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:35.625046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501a0da-964d-476a-9740-03262564d967 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:35.626722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 599/1191] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:35 2022] GET /identity/v3/projects/248bba28e2444036868485767be8fddb => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:35.636032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:35.739480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31268561-062a-4f4f-9387-7470cbc2f7d8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:35.748241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:35.775859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694aa1d3-de6b-4fa8-9693-926aa1bbc207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:35.776740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694aa1d3-de6b-4fa8-9693-926aa1bbc207 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:35.777017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694aa1d3-de6b-4fa8-9693-926aa1bbc207 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:35.777284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694aa1d3-de6b-4fa8-9693-926aa1bbc207 None admin] PATH_INFO: `/v3/projects/248bba28e2444036868485767be8fddb/users/b19710ac269845e5af1754dc661f26b8/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:35.779376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694aa1d3-de6b-4fa8-9693-926aa1bbc207 None admin] RBAC: Authorizing `identity:create_grant(project_id=248bba28e2444036868485767be8fddb, user_id=b19710ac269845e5af1754dc661f26b8, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:35.780822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694aa1d3-de6b-4fa8-9693-926aa1bbc207 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:35.795702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 593/1192] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:28:35 2022] PUT /identity/v3/projects/248bba28e2444036868485767be8fddb/users/b19710ac269845e5af1754dc661f26b8/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:28:37.927272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa8344-dc62-43b5-9df8-d7aa79175198 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:37.928112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa8344-dc62-43b5-9df8-d7aa79175198 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:37.928112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa8344-dc62-43b5-9df8-d7aa79175198 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:37.930364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 600/1193] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:28:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:28:37.936750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:37.937170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:37.937358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:37.963367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:28:37.964564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:28:38.043043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 594/1194] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:28:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:28:38.067295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa8344-dc62-43b5-9df8-d7aa79175198 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:38.127201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:38.181887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa8344-dc62-43b5-9df8-d7aa79175198 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:38.218328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:38.219555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:38.219915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:38.220280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:38.233747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:28:38.241342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:28:38.242376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:28:38.243402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 601/1195] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:28:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:28:38.248683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:38.348369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:38.352712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481d013-ec58-45cc-a2ee-c42e353c98a1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:38.391267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:38.391931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:38.392204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:38.392464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:38.393211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:38.393508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:38.394901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:38.403353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 595/1196] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:28:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:38.409319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:38.516626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc49884-f52d-4bac-8726-1f21909f4803 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:38.529565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:38.551494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:38.552354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:38.552631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:38.552901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] PATH_INFO: `/v3/users/b19710ac269845e5af1754dc661f26b8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:38.554087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] RBAC: Authorizing `identity:get_user(user_id=b19710ac269845e5af1754dc661f26b8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:38.555621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:38.557253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 602/1197] 10.222.0.32 () {56 vars in 1177 bytes} [Wed May 25 14:28:38 2022] GET /identity/v3/users/b19710ac269845e5af1754dc661f26b8 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:38.563211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:38.664396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332351a-0089-4213-bd5d-a1de3535e02b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:38.707326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:38.707971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:38.708280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:38.708556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:38.736827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:38.780070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:38.781524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:38.882125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 596/1198] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:28:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:28:38.891142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:38.996520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38432f-ebc4-4f09-b582-836979d5f7fc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:39.033861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcc852-1a41-4f83-8657-b85f171acfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:39.034721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcc852-1a41-4f83-8657-b85f171acfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:39.035031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcc852-1a41-4f83-8657-b85f171acfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:39.035303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcc852-1a41-4f83-8657-b85f171acfa3 None admin] PATH_INFO: `/v3/projects/248bba28e2444036868485767be8fddb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:39.036607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcc852-1a41-4f83-8657-b85f171acfa3 None admin] RBAC: Authorizing `identity:get_project(project_id=248bba28e2444036868485767be8fddb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:39.038034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcc852-1a41-4f83-8657-b85f171acfa3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:39.039805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 603/1199] 10.222.0.32 () {56 vars in 1186 bytes} [Wed May 25 14:28:38 2022] GET /identity/v3/projects/248bba28e2444036868485767be8fddb => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:39.046494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:28:39.108771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:39.153537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f4174-22c6-4af2-bbec-79ab83a3b784 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:28:39.189730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:28:39.190369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:28:39.190643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:28:39.190943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:28:39.191775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=248bba28e2444036868485767be8fddb, user.id=b19710ac269845e5af1754dc661f26b8) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:28:39.192076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:28:39.193511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83907f9a-5fa3-4282-baaf-e005cda046d1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:28:39.199494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 597/1200] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:28:39 2022] GET /identity/v3/role_assignments?user.id=b19710ac269845e5af1754dc661f26b8&scope.project.id=248bba28e2444036868485767be8fddb&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:28:49.028487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=76 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:28:49.196694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=38 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:39.393957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c610fbb-9b93-4a91-ba51-4aac50118303 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:39.394644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c610fbb-9b93-4a91-ba51-4aac50118303 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:39.394644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c610fbb-9b93-4a91-ba51-4aac50118303 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:39.396208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 604/1201] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:29:39.402935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:39.403632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:39.403632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:39.425010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:29:39.426331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:29:39.500731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 598/1202] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:29:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:29:39.526166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c610fbb-9b93-4a91-ba51-4aac50118303 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:39.656864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c610fbb-9b93-4a91-ba51-4aac50118303 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:39.694943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:39.696075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:39.696354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:39.696600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:39.710095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:39.711588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:39.730713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:39.732002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:39.733289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:39.733289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:39.733289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:39.733289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:39.733289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 605/1203] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:29:39.738480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:39.870228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bece27-f433-4dfc-a7d6-a7c0930b9c6f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:39.903377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:39.912643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:39.913295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:39.913544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:39.913811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:39.914549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:39.914896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:39.916605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0c2a6-1643-48ee-9342-73648f03160f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:39.932553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 599/1204] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:29:39.938215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:40.002129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:40.060287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89a3b3-c460-41b1-a1d0-26f22dcc6329 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:40.100353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dd89c-0ef1-45e2-8ec0-da04492ed26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:40.101047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dd89c-0ef1-45e2-8ec0-da04492ed26b None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:40.101297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dd89c-0ef1-45e2-8ec0-da04492ed26b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:40.101601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dd89c-0ef1-45e2-8ec0-da04492ed26b None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:40.102318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dd89c-0ef1-45e2-8ec0-da04492ed26b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:40.103762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0dd89c-0ef1-45e2-8ec0-da04492ed26b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:40.126445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 606/1205] 10.222.0.32 () {60 vars in 1132 bytes} [Wed May 25 14:29:39 2022] POST /identity/v3/users => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:29:42.573246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881efbd-c192-411c-9acf-32202f2ad4da None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:42.573834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881efbd-c192-411c-9acf-32202f2ad4da None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:42.573974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881efbd-c192-411c-9acf-32202f2ad4da None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:42.575609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 600/1206] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:29:42.581034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:42.581482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:42.581598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:42.600909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:29:42.602283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:29:42.679940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 607/1207] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:29:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:29:42.707028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881efbd-c192-411c-9acf-32202f2ad4da None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:42.826532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881efbd-c192-411c-9acf-32202f2ad4da None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:42.865876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:42.866929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:42.867242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:42.867554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:42.878697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:29:42.886839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:42.887854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:42.888701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 601/1208] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:29:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:42.897233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:42.938812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:43.005615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de84c60-3aed-4709-81cf-b002dfb065ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:43.046468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:43.047381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:43.047500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:43.047671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:43.048428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:43.048728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:43.050078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:43.063274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 608/1209] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:29:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:43.069751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:43.127998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:43.185613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0af29-581d-4084-81ff-b34549def7ba None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:43.234264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:43.235132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:43.235413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:43.235738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:43.250173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:43.251744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:43.271009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:43.272011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:43.273388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:43.273388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:43.273388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:43.273388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:43.273388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 602/1210] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:43.278528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:43.400393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:43.402247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e18fc0-1577-4009-a7b2-1a22573f8d99 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:43.443346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:43.444068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:43.444371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:43.444638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:43.445401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:43.445705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:43.447100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:43.459557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 609/1211] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:43.467735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:43.588343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9610f-a12b-45b6-a578-81dce13ce04f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:43.590379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:43.626956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:43.628026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:43.628361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:43.628619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:43.637023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:43.638516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:29:43.655176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:43.661222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:43.662430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 603/1212] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:43.664402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:43.771402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72f63-4848-45cb-9527-8b44413f7e05 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:43.813571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31877558-e256-462c-bb5c-76554605b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:43.814245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31877558-e256-462c-bb5c-76554605b439 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:43.814521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31877558-e256-462c-bb5c-76554605b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:43.814999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31877558-e256-462c-bb5c-76554605b439 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:43.815790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31877558-e256-462c-bb5c-76554605b439 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:43.816147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31877558-e256-462c-bb5c-76554605b439 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:43.817602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31877558-e256-462c-bb5c-76554605b439 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:43.836037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 610/1213] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 25 14:29:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:43.841923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:43.948055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245b080-22b0-4a80-921a-89acb1e42ce3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:43.982268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:43.982959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:43.983402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:43.983510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:44.020152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:44.061974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:44.063586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:44.176021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 604/1214] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:29:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:29:44.180860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31877558-e256-462c-bb5c-76554605b439 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:44.203305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:44.300819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31877558-e256-462c-bb5c-76554605b439 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:44.340629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:44.341720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:44.342104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:44.342523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:44.354726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:44.356355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:44.375041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:44.376207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:44.377329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:44.377329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:44.377329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:44.377329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:44.379029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 611/1215] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:44.384361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:44.469684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:44.517491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb675a0-d782-40a8-855c-65fdcf625591 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:44.565116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:44.566018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:44.566427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:44.566805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:44.567654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:44.568058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:44.569567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:44.585527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 605/1216] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:29:44.593122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:44.671034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:44.707451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe61b23-17cf-412e-857b-e578e652bb62 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:44.742277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:44.742976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:44.743366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:44.743818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:44.751447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:44.753020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:29:44.767183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:44.768252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:44.769401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:44.774838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 612/1217] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:44.776630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:44.885907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c0506-b484-4d7b-81cc-e030f8e06688 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:44.912336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:44.925534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:44.926191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:44.926471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:44.926744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:44.927562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:44.927866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:44.929321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eab913-5d78-4476-8c45-d7d78e37dec1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:44.941887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 606/1218] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:44.949081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:45.052414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7abce1-bf6b-49e9-a27d-84e8da8a7c03 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:45.085988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:45.094301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:45.095143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:45.095487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:45.095791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:45.096739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=7c1699703bdb4d2e8dac40a28792cd32) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:45.097097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:45.098520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bacf0c-2ac1-4f6c-a3b6-f6c29a0cbc26 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:45.104370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 613/1219] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:29:44 2022] GET /identity/v3/role_assignments?user.id=7c1699703bdb4d2e8dac40a28792cd32&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:29:47.369905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32f7bd-4b30-4512-b57d-7a022591b6a1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:47.370898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32f7bd-4b30-4512-b57d-7a022591b6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:47.370898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32f7bd-4b30-4512-b57d-7a022591b6a1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:47.371855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 607/1220] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:29:47.378051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:47.378550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:47.378651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:47.401097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:29:47.402271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:29:47.482570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 614/1221] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:29:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:29:47.507197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32f7bd-4b30-4512-b57d-7a022591b6a1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:47.622649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32f7bd-4b30-4512-b57d-7a022591b6a1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:47.663249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:47.664128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:47.664429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:47.664670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:47.675449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:29:47.683149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:47.684138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:47.685047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 608/1222] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:29:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:47.690952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:47.795252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d5f8a-3136-43d4-bf4b-7597f75a0b14 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:47.822720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:47.836163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:47.836867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:47.837166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:47.837467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:47.838308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:47.838641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:47.840231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:47.849839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 615/1223] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:29:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:47.855774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:47.900749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:47.961835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475cc0-baa1-4149-8d2e-9a2a6a8e8daa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:47.995663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:47.996517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:47.996815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:47.997118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:48.008135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:48.009849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:48.029191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:48.030254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:48.031327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:48.031327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:48.031327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:48.031327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:48.031327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 609/1224] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:48.036365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:48.140463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e9d7f-a526-49c3-8916-a692db224f8d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:48.179247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:48.179862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:48.180153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:48.180410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:48.181148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:48.181454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:48.184836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:48.197908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 616/1225] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:48.203895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:48.303690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:48.312302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e3301-b5fd-41bf-97ac-6c8aabbf5e85 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:48.348142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:48.348952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:48.349232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:48.349506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:48.356469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:48.357926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:29:48.372489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:48.374166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:48.375477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 610/1226] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:29:48.379805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:48.435298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:48.496966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe9836-2938-43df-af29-afc6edf7a87f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:48.535718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:48.536396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:48.536675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:48.536978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:48.537789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:48.538093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:48.539555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:48.557842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 617/1227] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 25 14:29:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:48.563608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:48.668567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca323d4c-736d-4f98-8c5f-5012ad4acbbe None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:48.705122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:48.705885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:48.706204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:48.706475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:48.717227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:48.718684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:48.739034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:48.740162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:48.741098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:48.741098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:48.741098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:48.741098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:48.741098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 611/1228] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:48.746622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:48.859682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a28f8-21fc-4cfd-8f6e-558d9035123d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:48.862151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:48.897676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:48.898328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:48.898603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:48.898906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:48.899632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:48.899935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:48.901307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:48.913448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 618/1229] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:48.919690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:48.929347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:49.033494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463739d9-8fba-4256-b745-d3cfcab68a30 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:49.073259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:49.074060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:49.074337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:49.074609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:49.082920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:49.084571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:29:49.099882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:49.101392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:49.102723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 612/1230] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:29:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:49.107182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:49.215644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53076-afa8-42ab-9f0e-9cf86f1e8ea6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:49.252687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:49.253398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:49.253694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:49.253990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:49.254933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:49.255260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:49.256838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea629bd-5607-438d-8cc1-2f8f690db6a9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:49.275761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 619/1231] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:29:49.282186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:49.356059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:49.394617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8dafe-4f23-44d5-b68d-ccf26915e746 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:49.437449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafcd6a7-dd47-40dd-80cd-50adecda6dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:49.438663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcd6a7-dd47-40dd-80cd-50adecda6dd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:49.438967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcd6a7-dd47-40dd-80cd-50adecda6dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:49.439231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafcd6a7-dd47-40dd-80cd-50adecda6dd9 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/7c1699703bdb4d2e8dac40a28792cd32/roles/4b9133eb9de74ba5bceca230df6a1aad` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:49.455598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcd6a7-dd47-40dd-80cd-50adecda6dd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=7c1699703bdb4d2e8dac40a28792cd32, role_id=4b9133eb9de74ba5bceca230df6a1aad)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:49.457396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafcd6a7-dd47-40dd-80cd-50adecda6dd9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:49.472238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 613/1232] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:29:49 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/7c1699703bdb4d2e8dac40a28792cd32/roles/4b9133eb9de74ba5bceca230df6a1aad => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:29:51.699592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ca2b-5207-4ada-948b-ab8bd9f01034 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:51.700185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ca2b-5207-4ada-948b-ab8bd9f01034 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:51.700185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ca2b-5207-4ada-948b-ab8bd9f01034 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:51.701680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 620/1233] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:29:51.707574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:51.708167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:51.708167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:51.725383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:29:51.726583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:29:51.794914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 614/1234] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:29:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:29:51.818426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300ca2b-5207-4ada-948b-ab8bd9f01034 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:51.843382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:51.923948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300ca2b-5207-4ada-948b-ab8bd9f01034 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:51.959130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:51.959850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:51.960106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:51.960390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:51.971046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:29:51.979271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:51.980432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 14:29:51.981455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:51.985565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 621/1235] 10.222.0.32 () {56 vars in 1102 bytes} [Wed May 25 14:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:51.986552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:52.034569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:52.088615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ef9f0-57af-4e6d-8125-d17a014849dc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:52.125289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:52.125965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:52.126248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:52.126523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:52.127313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:52.127622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:52.129014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:52.137148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 615/1236] 10.222.0.32 () {56 vars in 1103 bytes} [Wed May 25 14:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:52.143199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:52.234775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:52.253702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0f57f-e57a-464c-9853-e9661f665ddf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:52.292247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:52.293049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:52.293325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:52.293590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:52.304734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:52.306221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:52.324405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:52.325874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:52.327216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:52.327216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:52.327216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:52.327216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:52.330607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 622/1237] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:52.331685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:52.449135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b8d63-6f74-47ee-801d-b421487f34aa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:52.459929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:52.486673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:52.487428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:52.487771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:52.488046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:52.488888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:52.489311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:52.490982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:52.504386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 616/1238] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:52.510212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:52.630976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d3e95-97c0-4c73-9d43-ef137b089a17 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:52.667955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:52.668872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:52.669276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:52.669570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:52.672764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:52.678650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:52.680201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:29:52.696321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:52.697419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:52.698529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 623/1239] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:29:52.704996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:52.818176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdfe1a-81e5-4fad-90c2-34a7eda3f02e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:52.852565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:52.853271 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:52.853547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:52.853817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:52.854662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:52.854998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:52.856438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:52.875313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 617/1240] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 25 14:29:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:52.881336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:52.993254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06997728-bb33-4767-b6f0-5fada67e694d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:53.029936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:53.030605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:53.030899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:53.031172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:53.094073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:53.101894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:53.103509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:53.208332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 624/1241] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:29:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:29:53.214585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:53.243498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:53.331620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121ee1-95d1-4eae-a541-7a92928772b4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:53.367300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:53.368028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:53.368301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:53.368566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:53.378996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:53.380456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:53.399491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:53.400685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:53.401558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:53.401558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:53.401558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:53.401558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:53.401558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 618/1242] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:53.407111 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:53.511306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:53.517725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb23995-f9bd-4e55-8135-4a88526c64e5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:53.555568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:53.556224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:53.556551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:53.556864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:53.557588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:53.557956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:53.559485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:53.572254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 625/1243] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:29:53.578343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:53.661698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:53.689753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2358-f186-4c2a-a5f2-737d5435cd8d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:53.731520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:53.732507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:53.732865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:53.733169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:53.741281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:53.743229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:29:53.758308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:53.759300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:53.760370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 619/1244] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 14:29:53.765765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:53.877847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1f2b3-9063-4285-a22e-a99a3abed8d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:53.917270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:53.917999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:53.918139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:53.918412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:53.919249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:53.919554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:53.920992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e13b8a-4a51-4b3b-bdce-ea67c2eaf1c8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:53.936991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 626/1245] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:53.943748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:54.057283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf104d83-d37a-4b26-a3d5-5c0dec289f02 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:54.094731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:54.095420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:54.095765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:54.095962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:54.096933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] RBAC: Adding query filter params (role.id=4b9133eb9de74ba5bceca230df6a1aad, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=7c1699703bdb4d2e8dac40a28792cd32) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:54.097257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:54.098699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd632114-4669-436d-8854-0e867ed59fb3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:54.104964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 620/1246] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:29:53 2022] GET /identity/v3/role_assignments?user.id=7c1699703bdb4d2e8dac40a28792cd32&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=4b9133eb9de74ba5bceca230df6a1aad => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:56.489043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e819d-88fd-4bcf-91d1-94359c4f30a6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:56.489623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e819d-88fd-4bcf-91d1-94359c4f30a6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:56.489623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e819d-88fd-4bcf-91d1-94359c4f30a6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:56.491021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 627/1247] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:29:56.496956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:56.497418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:56.497545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:56.502998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:56.517358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:29:56.518844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:29:56.594081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 621/1248] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:29:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:29:56.620017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e819d-88fd-4bcf-91d1-94359c4f30a6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:56.639191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:56.733813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e819d-88fd-4bcf-91d1-94359c4f30a6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:56.768356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:56.769068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:56.769345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:56.769614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:56.781836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:29:56.790625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:56.797022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:56.798040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 628/1249] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:29:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:56.800604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:56.913633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f319b-3100-4e9e-b5fd-41a3c6db6dd0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:56.948904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:56.955012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:56.955831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:56.956148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:56.956419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:56.957244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:56.957643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:56.959353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:56.968996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 622/1250] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:29:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:56.975961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:57.065350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:57.091757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae7479-dbed-469a-995a-dee178f84d62 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:57.128699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:57.129558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:57.129868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:57.130136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:57.141179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:57.142680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:57.160676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:57.161624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:57.162506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:57.162506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:57.162506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:57.162506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:57.162506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 629/1251] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:57.169612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:57.280588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16726ae-56d5-4b5f-8471-815bc0cc1c96 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:57.321705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:57.322315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:57.322597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:57.322910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:57.323847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:57.324223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:57.325845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:57.339896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 623/1252] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:57.349528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:57.471465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894684d-5ab8-4d4e-9280-e3e665cbc0f2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:57.498900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:57.507230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:57.508075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:57.508394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:57.508722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:57.516654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:57.518112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:29:57.532857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:57.534237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:29:57.535395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 630/1253] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:57.540577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:57.600430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:57.648547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a3159-17d3-42de-b406-53071e4ea5c0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:57.687920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:57.688700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:57.689009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:57.689340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:57.690276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:57.690585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:57.692234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:57.710654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 624/1254] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 25 14:29:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:57.717057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:57.836325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbaab9-c89a-4761-9e2e-270570960814 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:57.873023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:57.873604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:57.873877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:57.874141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:57.930908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:57.943969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:57.945426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:58.048399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 631/1255] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:29:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:29:58.055116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:58.162401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f3084-d13c-4278-b397-1491b1ec3701 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:58.176685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:58.201015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:58.201801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:58.202079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:58.202347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:58.213754 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:58.215272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:29:58.234688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:58.235664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:58.236707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:29:58.236707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:29:58.236707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:29:58.236707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:58.236707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 625/1256] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:58.241866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:58.329888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:58.343834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1159b-3269-4059-9dcb-679413e350dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:58.382119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:58.383046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:58.383178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:58.383387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:58.384103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:58.384408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:58.385792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:58.398331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 632/1257] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:58.404239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:58.517390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927221b-d652-4da4-b471-381d85605ea0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:58.555797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:58.556621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:58.556943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:58.557263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:58.564107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:58.565629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:29:58.579411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:29:58.580731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:29:58.581642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 626/1258] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:29:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:29:58.589395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:58.696707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cb690-8600-456f-9e0c-b6836487f3c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:58.732098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785db222-2673-4427-8234-a5444188c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:58.732764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785db222-2673-4427-8234-a5444188c40b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:58.733033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785db222-2673-4427-8234-a5444188c40b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:58.733298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785db222-2673-4427-8234-a5444188c40b None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:58.734092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785db222-2673-4427-8234-a5444188c40b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:58.735925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785db222-2673-4427-8234-a5444188c40b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:58.735925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785db222-2673-4427-8234-a5444188c40b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:58.748619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 633/1259] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:29:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:29:58.754930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:29:58.787073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:29:58.868126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342df9af-8bcb-4758-b540-6be02d2b2cf3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:29:58.904094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:29:58.904784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:29:58.905112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:29:58.905347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:29:58.906236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=7c1699703bdb4d2e8dac40a28792cd32) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:29:58.906542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:29:58.908063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5810-f593-4e29-afaf-f6e27e188537 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:29:58.913983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 627/1260] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:29:58 2022] GET /identity/v3/role_assignments?user.id=7c1699703bdb4d2e8dac40a28792cd32&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:01.076385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b3661-2489-4f03-a6c6-de5639055d57 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:01.077196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b3661-2489-4f03-a6c6-de5639055d57 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:01.077196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b3661-2489-4f03-a6c6-de5639055d57 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:01.079309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 634/1261] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:30:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:30:01.086221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:01.086657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:01.086815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:01.113461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:30:01.114666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:30:01.191233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 628/1262] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:30:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:30:01.219813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b3661-2489-4f03-a6c6-de5639055d57 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:01.231569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:01.331198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b3661-2489-4f03-a6c6-de5639055d57 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:01.367794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:01.368584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:01.368862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:01.369130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:01.385067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:30:01.392673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:01.394207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:01.396026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 635/1263] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:30:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:01.401729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:01.425746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:01.513651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346da2ef-e60a-4681-aa83-469f0a4e295f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:01.548222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:01.548882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:01.549162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:01.549449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:01.550193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:01.550493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:01.551893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:01.559910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 629/1264] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:30:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:01.566116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:01.642229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:01.676741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e993-0daf-476c-b543-1ff207c0d68a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:01.714732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:01.715556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:01.715832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:01.716104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:01.727249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:01.728788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:30:01.746078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:01.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:01.748059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:30:01.748059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:30:01.748059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:01.748059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:01.748059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 636/1265] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:01.755920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:01.844842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:01.865001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a5e7f-e259-4f56-95dd-d4225284cbcf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:01.899548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:01.900130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:01.900406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:01.900669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:01.901386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:01.901693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:01.903150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:01.919316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 630/1266] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:01.927938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:02.042675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45396509-ad2b-43ab-bac9-b69d99373c5a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:02.063927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:02.081226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:02.082021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:02.082294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:02.082548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:02.090442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:02.091907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:30:02.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:02.107290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:02.108381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 637/1267] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:30:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:02.113560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:02.211955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec5430-8a98-482f-b466-4f0d796fa36b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:02.246147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:02.246795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:02.247102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:02.247371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:02.248198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:02.248501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:02.249969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:02.258611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:02.267712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 631/1268] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 25 14:30:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:02.273697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:02.373306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a27aa5-cc62-40f7-aa94-81e2fab76391 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:02.415096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:02.416111 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:02.416488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:02.416814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:02.431176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:02.432743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:30:02.450435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:02.451549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:02.452625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:30:02.452625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:30:02.452625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:02.452625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:02.452625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 638/1269] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:02.457578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:02.568833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25a7da-16c2-467a-8b02-445bfda72262 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:02.603653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:02.604301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:02.604581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:02.604853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:02.605568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:02.605875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:02.607317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:02.618689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 632/1270] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:02.624634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:02.658988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:02.729203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5107a2-0fcc-4ec7-a3fe-01a65e77828a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:02.772179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:02.772976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:02.773246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:02.773509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:02.780893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:02.782332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:30:02.797181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:02.798064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:02.799498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 639/1271] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:30:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:02.804662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:02.866584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:02.912166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04673e-dcf6-44f3-9c27-23f4f2a8e7ea None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:02.953422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:02.954080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:02.954359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:02.954633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:02.955454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:02.955757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:02.957202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd23fe6-a482-470a-98a8-f06b2c52d0a4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:02.969247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 633/1272] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:30:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:02.979022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:03.082525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:03.094158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421aae1b-9b36-470d-a47f-adb79a885009 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:03.131185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85156eb3-254b-4f8c-825e-a9d03bef7743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:03.132000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85156eb3-254b-4f8c-825e-a9d03bef7743 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:03.132279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85156eb3-254b-4f8c-825e-a9d03bef7743 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:03.132547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85156eb3-254b-4f8c-825e-a9d03bef7743 None admin] PATH_INFO: `/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/7c1699703bdb4d2e8dac40a28792cd32/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:03.134795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85156eb3-254b-4f8c-825e-a9d03bef7743 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab7ba398fd04278a753acc66664f6e1, user_id=7c1699703bdb4d2e8dac40a28792cd32, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:03.136215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85156eb3-254b-4f8c-825e-a9d03bef7743 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:03.151223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 640/1273] 10.222.0.32 () {58 vars in 1439 bytes} [Wed May 25 14:30:02 2022] PUT /identity/v3/projects/3ab7ba398fd04278a753acc66664f6e1/users/7c1699703bdb4d2e8dac40a28792cd32/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:30:05.296263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70386650-4123-4876-82f2-7b645cc5a07a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:05.296831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70386650-4123-4876-82f2-7b645cc5a07a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:05.296831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70386650-4123-4876-82f2-7b645cc5a07a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:05.298163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 634/1274] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:30:05.304690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:05.305377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:05.305527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:05.326462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:30:05.327662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:30:05.404424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 641/1275] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:30:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:30:05.438521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70386650-4123-4876-82f2-7b645cc5a07a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:05.543407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70386650-4123-4876-82f2-7b645cc5a07a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:05.562019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:05.583292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:05.584083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:05.584358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:05.584624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:05.595084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 14:30:05.602777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:05.603828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:05.604653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 635/1276] 10.222.0.32 () {56 vars in 1096 bytes} [Wed May 25 14:30:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:05.610365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:05.709325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:05.729067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6f07e-c033-4273-a52d-8df45eedddfb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:05.764879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:05.765529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:05.765802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:05.766072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:05.766807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:05.767134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:05.768536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:05.776605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 642/1277] 10.222.0.32 () {56 vars in 1099 bytes} [Wed May 25 14:30:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:05.782606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:05.887417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4039f6-5442-4757-9630-93ca03371457 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:05.922061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:05.922870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:05.923144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:05.923414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:05.934211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:05.935765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:30:05.952717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:05.953713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:05.954850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:30:05.954850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:30:05.954850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:05.954850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:05.954850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 636/1278] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:05.960602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:06.072457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f19d-d211-4808-a27e-75bc34314558 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:06.107309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:06.107963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:06.108243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:06.108513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:06.109266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:06.109574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:06.110961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:06.123273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 643/1279] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:06.131770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:06.162965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:06.233998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ba358-9c0e-4156-810d-091302e7983a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:06.269418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:06.270217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:06.270497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:06.270801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:06.279427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:06.280893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 14:30:06.295747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:06.297422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:06.298784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 637/1280] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:30:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:06.302985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:06.307100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:06.407452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cacee-75a9-45f3-b409-6ff2763d8a65 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:06.448141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:06.448750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:06.449030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:06.449306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:06.450166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:06.450469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:06.451951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:06.469053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 644/1281] 10.222.0.32 () {56 vars in 1143 bytes} [Wed May 25 14:30:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:06.477559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:06.573572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:06.585246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f3f4-c4a4-49e3-8dcd-7295836ee87b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:06.622030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:06.622691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:06.623042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:06.623320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:06.695117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:06.696573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:06.790171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:06.802006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 638/1282] 10.222.0.32 () {58 vars in 1282 bytes} [Wed May 25 14:30:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 14:30:06.808526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:06.879662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:06.936473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aac23c-e52f-45f1-bca3-cc45e8408bca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:06.978003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:06.978817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:06.979120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:06.979393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:06.990302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:06.991848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 14:30:07.012011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:07.013063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:07.014220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 14:30:07.014220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 14:30:07.014220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 14:30:07.014220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:07.014220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 645/1283] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:07.021250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:07.137882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a80a9b-a13a-4dbc-9819-009cf9f1b67c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:07.182123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:07.182791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:07.183690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:07.183968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:07.184722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:07.185024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:07.186426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:07.198236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 639/1284] 10.222.0.32 () {56 vars in 1109 bytes} [Wed May 25 14:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:07.204672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:07.337904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a34b-3e8a-4847-86d6-84dc73250d03 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:07.353721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:07.382423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:07.383260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:07.383538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:07.383806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:07.390366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:07.391772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 14:30:07.406019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:07.407158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 14:30:07.408311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 25 14:30:07.412686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 646/1285] 10.222.0.32 () {56 vars in 1111 bytes} [Wed May 25 14:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:07.414373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:07.467906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:07.535436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc655d1-0acd-44ba-9225-31f9d15bf70e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:07.574003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:07.574660 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:07.574966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:07.575239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:07.576028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:07.576337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:07.577780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658edcc-8dd6-49b1-964d-0e05a152796f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:07.589766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 640/1286] 10.222.0.32 () {56 vars in 1148 bytes} [Wed May 25 14:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:07.596574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:07.699554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ccbb2-61df-46a8-8ee1-c556013c70ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:07.734388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:07.735046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:07.735321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:07.735598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:07.736469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] RBAC: Adding query filter params (role.id=fa7c01e623754dfea0b39b9ee8f0d51d, scope.project.id=3ab7ba398fd04278a753acc66664f6e1, user.id=7c1699703bdb4d2e8dac40a28792cd32) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:07.736779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:07.738199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ee5e3-26fb-45ac-be3a-924b7026be68 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:07.744114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 647/1287] 10.222.0.32 () {56 vars in 1374 bytes} [Wed May 25 14:30:07 2022] GET /identity/v3/role_assignments?user.id=7c1699703bdb4d2e8dac40a28792cd32&scope.project.id=3ab7ba398fd04278a753acc66664f6e1&role.id=fa7c01e623754dfea0b39b9ee8f0d51d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:09.936877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c17e0-9509-44ab-be6c-a5108a2481c0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:09.937761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c17e0-9509-44ab-be6c-a5108a2481c0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:09.937761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c17e0-9509-44ab-be6c-a5108a2481c0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:09.940076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 641/1288] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:30:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:30:09.948037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:09.948479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:09.948567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:09.966359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:30:09.967608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:30:09.997791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:10.037422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 648/1289] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:30:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 14:30:10.060635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c17e0-9509-44ab-be6c-a5108a2481c0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:10.170426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c17e0-9509-44ab-be6c-a5108a2481c0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:10.181389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:10.215937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:10.216729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:10.217005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:10.217430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:10.218083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:10.219491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:10.231156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:10.232481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:10.233329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 642/1290] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:30:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:10.239217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:10.353866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d242df0-eccf-4352-bd8e-d804f751f757 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:10.388956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:10.389606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:10.389881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:10.390153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:10.390884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:10.391188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:10.392573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184f1db-58dc-4411-b452-3be2a6dd28c6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:10.397520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 649/1291] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 25 14:30:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:10.405561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:10.514505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47eb4-02ef-4319-9a68-2159ffcf2a76 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:10.553097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:10.553762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:10.554042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:10.554315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:10.555055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:10.555342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:10.556729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40669c3f-abc2-4575-a42e-67a079f78784 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:10.562147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 643/1292] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 25 14:30:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:30:12.728177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408511e9-76a4-4720-a738-4f0db53481be None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:12.728741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408511e9-76a4-4720-a738-4f0db53481be None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:12.728741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408511e9-76a4-4720-a738-4f0db53481be None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:12.730139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 650/1293] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:30:12.736639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c77ee-bc76-433f-9062-b1af0789d258 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:12.737403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c77ee-bc76-433f-9062-b1af0789d258 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:12.737527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c77ee-bc76-433f-9062-b1af0789d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:12.760839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7a4c77ee-bc76-433f-9062-b1af0789d258 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:30:12.762018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7a4c77ee-bc76-433f-9062-b1af0789d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:30:12.794363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:12.834379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 644/1294] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:30:12.857722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408511e9-76a4-4720-a738-4f0db53481be None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:12.893913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:12.967800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408511e9-76a4-4720-a738-4f0db53481be None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:13.003798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a971de0-2b53-46a2-8839-bff2f0dafa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:13.004388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a971de0-2b53-46a2-8839-bff2f0dafa39 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:13.004669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a971de0-2b53-46a2-8839-bff2f0dafa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:13.004942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a971de0-2b53-46a2-8839-bff2f0dafa39 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:13.005658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a971de0-2b53-46a2-8839-bff2f0dafa39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:13.007063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a971de0-2b53-46a2-8839-bff2f0dafa39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:13.024187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 651/1295] 10.222.0.32 () {60 vars in 1141 bytes} [Wed May 25 14:30:12 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:30:15.210054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96104b9f-8069-4452-9497-5319762410f4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:15.210605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96104b9f-8069-4452-9497-5319762410f4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:15.210605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96104b9f-8069-4452-9497-5319762410f4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:15.212601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 645/1296] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:30:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:30:15.221747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:15.222170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:15.222324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:15.245409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:30:15.246597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:30:15.322424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 652/1297] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:30:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:30:15.345881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96104b9f-8069-4452-9497-5319762410f4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:15.465420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96104b9f-8069-4452-9497-5319762410f4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:15.509062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:15.509802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:15.510087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:15.510364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] PATH_INFO: `/v3/services/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:15.511058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:15.512521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:15.524688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:15.525787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:15.532012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 646/1298] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:30:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:15.533051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:15.551770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:15.644673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab65e3-7fbd-41b7-b2b1-627a6ff81b75 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:15.680695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:15.681670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:15.682063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:15.682433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:15.683583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:15.683999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:15.686131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:15.692693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 653/1299] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 25 14:30:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:15.699086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:15.733526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:15.805272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19693c-b5bc-4544-afa9-e825c781caec None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:15.839830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:15.840481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:15.840764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:15.841037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:15.841834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] RBAC: Adding query filter params (interface=public, service_id=273767d932c345c0bd45295bd035bab4, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:15.842150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:15.843565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fd6de-2106-4b75-9722-340b3cebab8e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:15.848650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 647/1300] 10.222.0.32 () {56 vars in 1251 bytes} [Wed May 25 14:30:15 2022] GET /identity/v3/endpoints?service_id=273767d932c345c0bd45295bd035bab4&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:15.854655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:15.950206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:15.970905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3de6a-7ff7-48a1-bbbb-3b5bf1883b8a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:16.014311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9056b4c-1661-4640-9d7b-ed7b27ac08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:16.014915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9056b4c-1661-4640-9d7b-ed7b27ac08a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:16.015191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9056b4c-1661-4640-9d7b-ed7b27ac08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:16.015460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9056b4c-1661-4640-9d7b-ed7b27ac08a8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:16.016139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9056b4c-1661-4640-9d7b-ed7b27ac08a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:16.017557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9056b4c-1661-4640-9d7b-ed7b27ac08a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:16.024650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 654/1301] 10.222.0.32 () {56 vars in 1087 bytes} [Wed May 25 14:30:15 2022] GET /identity/v3/services => generated 2191 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:30:18.191068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b592a-e13d-426b-abaa-634ea247a93b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:18.191636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b592a-e13d-426b-abaa-634ea247a93b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:18.191636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b592a-e13d-426b-abaa-634ea247a93b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:18.193001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 648/1302] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:30:18.199403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:18.200143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:18.200274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:18.220922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:30:18.222096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:30:18.292737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 655/1303] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:30:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 14:30:18.317930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b592a-e13d-426b-abaa-634ea247a93b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:18.421771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:18.428930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b592a-e13d-426b-abaa-634ea247a93b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:18.464378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:18.465107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:18.465385 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:18.465651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:18.466284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:18.467708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 14:30:18.479824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 14:30:18.480983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 25 14:30:18.482066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 649/1304] 10.222.0.32 () {56 vars in 1117 bytes} [Wed May 25 14:30:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 14:30:18.487387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:18.569706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:18.602928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171fb17-f7e5-44e1-bb38-c1fd1a1f8d7d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:18.638688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:18.639456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:18.639754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:18.640044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:18.640756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:30:18.641057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:18.642654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2716d54-a863-4442-9b39-6b82aaf85b4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:18.648240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 656/1305] 10.222.0.32 () {56 vars in 1116 bytes} [Wed May 25 14:30:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:30:18.655146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:30:18.773092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05e1c8-44a7-4b36-9fd9-9dff0cbf95fe None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:30:18.808493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e6732-5fcc-43ea-8aad-a54f794c05e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:30:18.809151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e6732-5fcc-43ea-8aad-a54f794c05e6 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:30:18.809424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e6732-5fcc-43ea-8aad-a54f794c05e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:30:18.809715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e6732-5fcc-43ea-8aad-a54f794c05e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:30:18.810422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e6732-5fcc-43ea-8aad-a54f794c05e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:30:18.811826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e6732-5fcc-43ea-8aad-a54f794c05e6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:30:18.833293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 650/1306] 10.222.0.32 () {60 vars in 1144 bytes} [Wed May 25 14:30:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:30:28.645155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=30 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:30:28.828561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:22.654472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b055cb-45f8-482f-bd55-5cd88fa8dcb0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:22.655089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b055cb-45f8-482f-bd55-5cd88fa8dcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:22.655089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b055cb-45f8-482f-bd55-5cd88fa8dcb0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:22.656448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 657/1307] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:22.662166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8a84-facb-4fba-82ce-0b092b6b54d8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:22.662596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8a84-facb-4fba-82ce-0b092b6b54d8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:22.662834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8a84-facb-4fba-82ce-0b092b6b54d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:22.743132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-44bf8a84-facb-4fba-82ce-0b092b6b54d8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:22.744554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-44bf8a84-facb-4fba-82ce-0b092b6b54d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:22.912204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 651/1308] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:22.960960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49960d65-6c7c-49a9-ac55-068a5102f256 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:22.961352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49960d65-6c7c-49a9-ac55-068a5102f256 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:22.962012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49960d65-6c7c-49a9-ac55-068a5102f256 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:22.964170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 658/1309] 10.222.0.32 () {54 vars in 964 bytes} [Wed May 25 14:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:22.970622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca49ad3-2943-47b9-b5da-7c4652bf2469 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:22.970964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca49ad3-2943-47b9-b5da-7c4652bf2469 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:22.971247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca49ad3-2943-47b9-b5da-7c4652bf2469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:23.051308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9ca49ad3-2943-47b9-b5da-7c4652bf2469 None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:23.052483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9ca49ad3-2943-47b9-b5da-7c4652bf2469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:23.074271 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:23.155415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 652/1310] 10.222.0.32 () {58 vars in 1061 bytes} [Wed May 25 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:23.160697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49960d65-6c7c-49a9-ac55-068a5102f256 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:23.163199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-49960d65-6c7c-49a9-ac55-068a5102f256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:23.288587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49960d65-6c7c-49a9-ac55-068a5102f256 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:23.366630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:23.367280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:23.367552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:23.367802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:23.369419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:23.427662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:23.487835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:23.489293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2b18a-fe05-4c42-9913-3e25ca5e649a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:23.510706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 659/1311] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:31:25.989859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12732356-b724-42cf-884b-fbbc0bff8850 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:25.990376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12732356-b724-42cf-884b-fbbc0bff8850 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:25.990682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12732356-b724-42cf-884b-fbbc0bff8850 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:25.995283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 653/1312] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:25.999202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500a03-c91b-433f-8742-46d74cdfd8ae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:25.999654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500a03-c91b-433f-8742-46d74cdfd8ae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:26.000014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500a03-c91b-433f-8742-46d74cdfd8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:26.094086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-34500a03-c91b-433f-8742-46d74cdfd8ae None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:26.095766 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34500a03-c91b-433f-8742-46d74cdfd8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:26.202699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 660/1313] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:26.241469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12732356-b724-42cf-884b-fbbc0bff8850 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:26.243326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-12732356-b724-42cf-884b-fbbc0bff8850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:26.357620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12732356-b724-42cf-884b-fbbc0bff8850 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:26.438192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:26.441475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:26.442156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:26.442400 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:26.443062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:26.445149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:26.566653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:26.568152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631b4a3-570c-4e4e-982b-750ef345cae3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:26.589683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 654/1314] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:29.113152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607c8e-9b98-4088-8a0a-a83cfa013031 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:29.113699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607c8e-9b98-4088-8a0a-a83cfa013031 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:29.113699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26607c8e-9b98-4088-8a0a-a83cfa013031 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:29.115138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 661/1315] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:29.121408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084d5ee-013c-4519-841a-5a2e09c124b0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:29.122134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084d5ee-013c-4519-841a-5a2e09c124b0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:29.122276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084d5ee-013c-4519-841a-5a2e09c124b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:29.193286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8084d5ee-013c-4519-841a-5a2e09c124b0 None None] MFA Rules not processed for user `a1a9cfcf6d5443678b5f66674068c1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:29.194496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8084d5ee-013c-4519-841a-5a2e09c124b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:29.255666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:29.311057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 655/1316] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:29.374826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9226ce1-ed18-4fc5-8103-fdf3dfcf3133 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:29.375019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9226ce1-ed18-4fc5-8103-fdf3dfcf3133 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:29.375291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9226ce1-ed18-4fc5-8103-fdf3dfcf3133 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:29.376694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 662/1317] 10.222.0.32 () {54 vars in 964 bytes} [Wed May 25 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:29.382976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beacb6-a8aa-469c-96b5-341adad8adea None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:29.383264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beacb6-a8aa-469c-96b5-341adad8adea None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:29.383530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9beacb6-a8aa-469c-96b5-341adad8adea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:29.461629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a9beacb6-a8aa-469c-96b5-341adad8adea None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:29.462860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a9beacb6-a8aa-469c-96b5-341adad8adea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:29.539243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 656/1318] 10.222.0.32 () {58 vars in 1061 bytes} [Wed May 25 14:31:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:29.549056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9226ce1-ed18-4fc5-8103-fdf3dfcf3133 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:29.551478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a9226ce1-ed18-4fc5-8103-fdf3dfcf3133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:29.566863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:29.667747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9226ce1-ed18-4fc5-8103-fdf3dfcf3133 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:29.747143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:29.747779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:29.748054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:29.748316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:29.749942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:29.818112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:29.864772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:29.866202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b281c0a-ae4b-4e20-a3f2-cb35c284e20d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:29.886302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 663/1319] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:33.030705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c7a3-6e0a-4a90-a78e-d8aa965db21b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:33.031567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c7a3-6e0a-4a90-a78e-d8aa965db21b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:33.031567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3c7a3-6e0a-4a90-a78e-d8aa965db21b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:33.033245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 657/1320] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:33.039070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8b61-15a8-41ff-a474-41f2fe96e028 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:33.039521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8b61-15a8-41ff-a474-41f2fe96e028 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:33.039614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab8b61-15a8-41ff-a474-41f2fe96e028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:33.103791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1eab8b61-15a8-41ff-a474-41f2fe96e028 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:33.105008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1eab8b61-15a8-41ff-a474-41f2fe96e028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:33.206432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 664/1321] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:33.245196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3c7a3-6e0a-4a90-a78e-d8aa965db21b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:33.247684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cce3c7a3-6e0a-4a90-a78e-d8aa965db21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:33.307634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:33.363204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3c7a3-6e0a-4a90-a78e-d8aa965db21b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:33.453483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:33.454263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:33.454545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:33.454850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:33.456518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:33.567596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:33.582467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:33.583994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69761ce2-94a5-4f99-9527-49f76c5a312f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:33.603562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 658/1322] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:36.573614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377757a-9b75-40ac-b21e-5a5d73d15d0a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:36.574174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377757a-9b75-40ac-b21e-5a5d73d15d0a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:36.574174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377757a-9b75-40ac-b21e-5a5d73d15d0a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:36.575592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 665/1323] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:36.581120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bfd45-daf7-4d5e-ad75-e266b0d35236 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:36.581563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bfd45-daf7-4d5e-ad75-e266b0d35236 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:36.581674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bfd45-daf7-4d5e-ad75-e266b0d35236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:36.647836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-266bfd45-daf7-4d5e-ad75-e266b0d35236 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:36.649046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-266bfd45-daf7-4d5e-ad75-e266b0d35236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:36.747004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 659/1324] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:36.782884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377757a-9b75-40ac-b21e-5a5d73d15d0a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:36.784648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d377757a-9b75-40ac-b21e-5a5d73d15d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:36.805395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:36.892521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377757a-9b75-40ac-b21e-5a5d73d15d0a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:36.968495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:36.969144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:36.969424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:36.969726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:36.971405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:37.045504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:37.086386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:37.087868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92597216-4c10-4307-b2df-402f520d49d9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:37.106079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 666/1325] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:39.346874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603a98b-ec58-4052-9aa6-2433c3b6d21c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:39.347416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603a98b-ec58-4052-9aa6-2433c3b6d21c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:39.347416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603a98b-ec58-4052-9aa6-2433c3b6d21c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:39.348897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 660/1326] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:39.354426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087e2a8-5782-4681-89b8-ef6f5aa57720 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:39.354863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087e2a8-5782-4681-89b8-ef6f5aa57720 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:39.355114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8087e2a8-5782-4681-89b8-ef6f5aa57720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:39.425506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8087e2a8-5782-4681-89b8-ef6f5aa57720 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:39.427018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8087e2a8-5782-4681-89b8-ef6f5aa57720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:39.525778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 667/1327] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:39.561075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603a98b-ec58-4052-9aa6-2433c3b6d21c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:39.562708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5603a98b-ec58-4052-9aa6-2433c3b6d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:39.627852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:39.676681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603a98b-ec58-4052-9aa6-2433c3b6d21c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:39.753223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:39.753870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:39.754152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:39.754427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:39.756219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:39.867132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:39.869527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:39.871541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea542b9f-707d-444c-96b0-c628cd29fa9b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:39.890193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 661/1328] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:42.135315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5bf7-3439-423c-8a18-7de4f21cb319 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:42.135881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5bf7-3439-423c-8a18-7de4f21cb319 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:42.135881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5bf7-3439-423c-8a18-7de4f21cb319 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:42.137267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 668/1329] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:42.142741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d2a1a-a91b-403d-94b1-969f405e92bd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:42.143191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d2a1a-a91b-403d-94b1-969f405e92bd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:42.143448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d2a1a-a91b-403d-94b1-969f405e92bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:42.210679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1d7d2a1a-a91b-403d-94b1-969f405e92bd None None] MFA Rules not processed for user `a1a9cfcf6d5443678b5f66674068c1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:42.211941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d7d2a1a-a91b-403d-94b1-969f405e92bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:42.322193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 662/1330] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:42.359087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5bf7-3439-423c-8a18-7de4f21cb319 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:42.360686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8e3c5bf7-3439-423c-8a18-7de4f21cb319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:42.392596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:42.467953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5bf7-3439-423c-8a18-7de4f21cb319 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:42.544287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:42.544937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:42.545224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:42.545475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:42.547131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:42.624540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:42.656486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:42.657913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615aaedf-6388-481e-b1c4-baed983f84dc service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:42.676217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 669/1331] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:45.084690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac826a-f426-4f82-8c0f-aee8bc2168fb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:45.085483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac826a-f426-4f82-8c0f-aee8bc2168fb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:45.085483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac826a-f426-4f82-8c0f-aee8bc2168fb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:45.087493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 663/1332] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:45.096284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920b1fe-99ee-477a-81df-75c3ee441441 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:45.096711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920b1fe-99ee-477a-81df-75c3ee441441 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:45.096829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920b1fe-99ee-477a-81df-75c3ee441441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:45.168534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0920b1fe-99ee-477a-81df-75c3ee441441 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:45.169780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0920b1fe-99ee-477a-81df-75c3ee441441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:45.275318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 670/1333] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:45.314867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac826a-f426-4f82-8c0f-aee8bc2168fb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:45.317323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23ac826a-f426-4f82-8c0f-aee8bc2168fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:45.394593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:45.430114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac826a-f426-4f82-8c0f-aee8bc2168fb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:45.520321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:45.520987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:45.521256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:45.521521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:45.523288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:45.641170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:45.642682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1118c1-f3b9-452c-8f0f-f9cdf5e6b65b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:45.653632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:45.664145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 664/1334] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:47.901225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fb97f-ddc1-4352-badf-054be833d385 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:47.901782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fb97f-ddc1-4352-badf-054be833d385 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:47.901782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fb97f-ddc1-4352-badf-054be833d385 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:47.903259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 671/1335] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:47.908688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca675-b1fb-412d-8291-f3635c3547a0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:47.909127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca675-b1fb-412d-8291-f3635c3547a0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:47.909219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca675-b1fb-412d-8291-f3635c3547a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:47.977206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ee6ca675-b1fb-412d-8291-f3635c3547a0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:47.978399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ee6ca675-b1fb-412d-8291-f3635c3547a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:48.092173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 665/1336] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:48.135171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fb97f-ddc1-4352-badf-054be833d385 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:48.137561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-070fb97f-ddc1-4352-badf-054be833d385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:48.182022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:48.256028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fb97f-ddc1-4352-badf-054be833d385 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:48.335523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:48.336166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:48.336444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:48.336707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:48.338341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:48.433541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:48.460189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:48.461634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cbb07-9902-4d0d-9fe5-d9bd12361361 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:48.480158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 672/1337] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:51.015243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7b55f-ca02-4064-819c-83bd8e787381 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:51.016103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7b55f-ca02-4064-819c-83bd8e787381 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:51.016103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7b55f-ca02-4064-819c-83bd8e787381 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:51.018318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 666/1338] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:51.026697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3fa3-78d3-4897-b5bb-f271d7d6549a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:51.027287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3fa3-78d3-4897-b5bb-f271d7d6549a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:51.027391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3fa3-78d3-4897-b5bb-f271d7d6549a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:51.099914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0e8e3fa3-78d3-4897-b5bb-f271d7d6549a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:51.102383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0e8e3fa3-78d3-4897-b5bb-f271d7d6549a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:51.210315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 673/1339] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:31:51.248915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7b55f-ca02-4064-819c-83bd8e787381 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:51.250483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5e7b55f-ca02-4064-819c-83bd8e787381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:51.337302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:51.374838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7b55f-ca02-4064-819c-83bd8e787381 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:51.448488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:51.449140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:51.449393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:51.449679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:51.451354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:51.559393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:51.560829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e766e7-86e6-4158-8c50-e8cf8ec13e7b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:51.578974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 667/1340] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:55.923459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcb13f-d5a3-4ee2-8474-b85cb7178242 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:55.923982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcb13f-d5a3-4ee2-8474-b85cb7178242 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:55.923982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcb13f-d5a3-4ee2-8474-b85cb7178242 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:55.925597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 674/1341] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:55.931117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e04a4-f867-4dfb-b3ee-47ffc468f505 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:55.931584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e04a4-f867-4dfb-b3ee-47ffc468f505 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:55.931671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e04a4-f867-4dfb-b3ee-47ffc468f505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:55.937984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:56.004632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-431e04a4-f867-4dfb-b3ee-47ffc468f505 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:56.005841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-431e04a4-f867-4dfb-b3ee-47ffc468f505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:56.115048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 668/1342] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:56.154246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcb13f-d5a3-4ee2-8474-b85cb7178242 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:56.156501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34dcb13f-d5a3-4ee2-8474-b85cb7178242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:56.209528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:31:56.258630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcb13f-d5a3-4ee2-8474-b85cb7178242 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:56.262555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950ec14-b8fc-48fd-b068-0eedbd7d83dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:56.263231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ec14-b8fc-48fd-b068-0eedbd7d83dd service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:56.263505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ec14-b8fc-48fd-b068-0eedbd7d83dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:56.263759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950ec14-b8fc-48fd-b068-0eedbd7d83dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:56.273462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950ec14-b8fc-48fd-b068-0eedbd7d83dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:56.274934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2950ec14-b8fc-48fd-b068-0eedbd7d83dd service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:56.285113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 675/1343] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:31:59.078337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aeeb7-2aaa-48df-bb7a-ee1122b416f9 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:59.078893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aeeb7-2aaa-48df-bb7a-ee1122b416f9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:59.078893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7aeeb7-2aaa-48df-bb7a-ee1122b416f9 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:59.080206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 669/1344] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:31:59.085720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902452a-b522-4e96-a6da-73178ba875e6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:59.086127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902452a-b522-4e96-a6da-73178ba875e6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:59.086303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902452a-b522-4e96-a6da-73178ba875e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:59.115527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6902452a-b522-4e96-a6da-73178ba875e6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:31:59.116706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6902452a-b522-4e96-a6da-73178ba875e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:59.198585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 676/1345] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:31:59.235317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aeeb7-2aaa-48df-bb7a-ee1122b416f9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:31:59.237448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6f7aeeb7-2aaa-48df-bb7a-ee1122b416f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:31:59.252361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7aeeb7-2aaa-48df-bb7a-ee1122b416f9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:31:59.256290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc1ae5-ffd1-4cf4-a3ec-1c77b2563ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:31:59.256938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc1ae5-ffd1-4cf4-a3ec-1c77b2563ebd service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:31:59.257212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc1ae5-ffd1-4cf4-a3ec-1c77b2563ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:31:59.257483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc1ae5-ffd1-4cf4-a3ec-1c77b2563ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:31:59.266848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc1ae5-ffd1-4cf4-a3ec-1c77b2563ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:31:59.268298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc1ae5-ffd1-4cf4-a3ec-1c77b2563ebd service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:31:59.278082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 670/1346] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:31:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:03.411871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3c4cb-f06c-465f-8fd1-c29b7649cd5c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:03.412393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3c4cb-f06c-465f-8fd1-c29b7649cd5c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:03.412393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3c4cb-f06c-465f-8fd1-c29b7649cd5c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:03.413856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 677/1347] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:03.419364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea97e8-0b26-4ba9-9b9f-345caddc5242 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:03.419811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea97e8-0b26-4ba9-9b9f-345caddc5242 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:03.419892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea97e8-0b26-4ba9-9b9f-345caddc5242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:03.447318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c7ea97e8-0b26-4ba9-9b9f-345caddc5242 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:03.447318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c7ea97e8-0b26-4ba9-9b9f-345caddc5242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:03.456197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 671/1348] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:03.494125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3c4cb-f06c-465f-8fd1-c29b7649cd5c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:03.506357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3c4cb-f06c-465f-8fd1-c29b7649cd5c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:03.510422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7eb3ed-5091-442f-a8cf-ca14d1585d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:03.511173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eb3ed-5091-442f-a8cf-ca14d1585d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:03.511521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eb3ed-5091-442f-a8cf-ca14d1585d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:03.511866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eb3ed-5091-442f-a8cf-ca14d1585d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:03.521940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7eb3ed-5091-442f-a8cf-ca14d1585d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:03.523486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7eb3ed-5091-442f-a8cf-ca14d1585d87 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:03.533538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 678/1349] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:08.988069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeab892-a90f-426e-924a-2c52bf9c076b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:08.988986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeab892-a90f-426e-924a-2c52bf9c076b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:08.989442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeab892-a90f-426e-924a-2c52bf9c076b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:08.991239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 672/1350] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:08.996890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0daf8-f330-4a5c-bc55-42e7f043c464 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:08.997726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0daf8-f330-4a5c-bc55-42e7f043c464 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:08.998147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0daf8-f330-4a5c-bc55-42e7f043c464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:09.119786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-90c0daf8-f330-4a5c-bc55-42e7f043c464 None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:09.121300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90c0daf8-f330-4a5c-bc55-42e7f043c464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:09.150277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:32:09.209016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 679/1351] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:32:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:32:13.244210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa869-a6b6-4b46-a9a3-745106bc6367 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:13.244725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa869-a6b6-4b46-a9a3-745106bc6367 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:13.244725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70daa869-a6b6-4b46-a9a3-745106bc6367 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:13.246245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 673/1352] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:13.251112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a9d1a-31de-4bc9-a3f0-204c5bcd333f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:13.251524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a9d1a-31de-4bc9-a3f0-204c5bcd333f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:13.251656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a9d1a-31de-4bc9-a3f0-204c5bcd333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:13.270803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d02a9d1a-31de-4bc9-a3f0-204c5bcd333f None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:13.272001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d02a9d1a-31de-4bc9-a3f0-204c5bcd333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:13.280979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 680/1353] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:13.440085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=72 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:32:14.802632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46647d9d-b380-422d-b3b1-42171ac36676 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:14.803415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46647d9d-b380-422d-b3b1-42171ac36676 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:14.803415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46647d9d-b380-422d-b3b1-42171ac36676 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:14.804836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 674/1354] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:14.810048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4010f4-db90-4e6a-854b-a72ddc074cc6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:14.810489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4010f4-db90-4e6a-854b-a72ddc074cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:14.810602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4010f4-db90-4e6a-854b-a72ddc074cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:14.829355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a4010f4-db90-4e6a-854b-a72ddc074cc6 None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:14.830553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a4010f4-db90-4e6a-854b-a72ddc074cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:14.839578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 681/1355] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:18.726032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401580ad-369b-4415-9f79-b369363a5690 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:18.727031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401580ad-369b-4415-9f79-b369363a5690 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:18.727417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401580ad-369b-4415-9f79-b369363a5690 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:18.729146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 675/1356] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:18.734595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d7e3-ec48-4dd2-ba09-b444cc24432b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:18.735058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d7e3-ec48-4dd2-ba09-b444cc24432b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:18.735233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d7e3-ec48-4dd2-ba09-b444cc24432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:18.753324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c87d7e3-ec48-4dd2-ba09-b444cc24432b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:18.754504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c87d7e3-ec48-4dd2-ba09-b444cc24432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:18.763929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 682/1357] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:18.819329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f800e2-7e3e-45be-a7a3-1e6f481e0f67 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:18.819844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f800e2-7e3e-45be-a7a3-1e6f481e0f67 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:18.820259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f800e2-7e3e-45be-a7a3-1e6f481e0f67 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:18.822713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 676/1358] 10.222.0.32 () {54 vars in 961 bytes} [Wed May 25 14:32:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:18.830129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63e754-bb4a-48eb-8dd7-85f1c9b98fd6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:18.830614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63e754-bb4a-48eb-8dd7-85f1c9b98fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:18.831161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63e754-bb4a-48eb-8dd7-85f1c9b98fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:18.883338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ec63e754-bb4a-48eb-8dd7-85f1c9b98fd6 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:18.884634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec63e754-bb4a-48eb-8dd7-85f1c9b98fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:18.984360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 683/1359] 10.222.0.32 () {58 vars in 1058 bytes} [Wed May 25 14:32:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:32:18.992103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f800e2-7e3e-45be-a7a3-1e6f481e0f67 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:19.006189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f800e2-7e3e-45be-a7a3-1e6f481e0f67 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:19.012212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca048ac-580f-40f6-b88a-2956dc0b484a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:19.013106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca048ac-580f-40f6-b88a-2956dc0b484a service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:19.013621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca048ac-580f-40f6-b88a-2956dc0b484a service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:19.014097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca048ac-580f-40f6-b88a-2956dc0b484a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:19.025519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca048ac-580f-40f6-b88a-2956dc0b484a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:19.027859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca048ac-580f-40f6-b88a-2956dc0b484a service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:19.039549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 677/1360] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:21.442626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec631a-6a96-4a81-8e34-e199049a040e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:21.443324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec631a-6a96-4a81-8e34-e199049a040e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:21.443324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec631a-6a96-4a81-8e34-e199049a040e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:21.444958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 684/1361] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:21.450572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cebe0-e676-4c9b-a868-3672f0002f2d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:21.451058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cebe0-e676-4c9b-a868-3672f0002f2d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:21.451162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cebe0-e676-4c9b-a868-3672f0002f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:21.470868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5c4cebe0-e676-4c9b-a868-3672f0002f2d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:21.472019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c4cebe0-e676-4c9b-a868-3672f0002f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:21.480953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 678/1362] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:21.525879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec631a-6a96-4a81-8e34-e199049a040e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:21.551742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec631a-6a96-4a81-8e34-e199049a040e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:21.557071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d5439-65b4-4642-af69-2ba2cc6612f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:21.557721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5439-65b4-4642-af69-2ba2cc6612f1 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:21.557994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5439-65b4-4642-af69-2ba2cc6612f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:21.558258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d5439-65b4-4642-af69-2ba2cc6612f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:21.567038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d5439-65b4-4642-af69-2ba2cc6612f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:21.568695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d5439-65b4-4642-af69-2ba2cc6612f1 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:21.578594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 685/1363] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:24.158446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beac204-95bf-47de-91bb-7cbeb3e1707b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:24.161643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beac204-95bf-47de-91bb-7cbeb3e1707b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:24.161643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beac204-95bf-47de-91bb-7cbeb3e1707b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:24.161643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 679/1364] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:24.166629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbeafc-030b-4abd-a382-05da4f154ce3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:24.167074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbeafc-030b-4abd-a382-05da4f154ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:24.167223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbeafc-030b-4abd-a382-05da4f154ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:24.187525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-60cbeafc-030b-4abd-a382-05da4f154ce3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:24.188996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60cbeafc-030b-4abd-a382-05da4f154ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:24.210967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 686/1365] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:24.250356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d490a1-0dfd-4336-879e-b69ddf98971e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:24.250664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d490a1-0dfd-4336-879e-b69ddf98971e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:24.251021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d490a1-0dfd-4336-879e-b69ddf98971e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:24.252592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 680/1366] 10.222.0.32 () {54 vars in 961 bytes} [Wed May 25 14:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:24.258584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b8124-25c0-4323-b6b8-5d5aa89f4ebf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:24.258970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b8124-25c0-4323-b6b8-5d5aa89f4ebf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:24.259305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b8124-25c0-4323-b6b8-5d5aa89f4ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:24.282659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e5b8124-25c0-4323-b6b8-5d5aa89f4ebf None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:24.283870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e5b8124-25c0-4323-b6b8-5d5aa89f4ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:24.294874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 687/1367] 10.222.0.32 () {58 vars in 1058 bytes} [Wed May 25 14:32:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:24.303204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d490a1-0dfd-4336-879e-b69ddf98971e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:24.319020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d490a1-0dfd-4336-879e-b69ddf98971e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:24.324721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f669b-0e3c-4c27-bc6e-b900da138e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:24.325405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f669b-0e3c-4c27-bc6e-b900da138e46 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:24.325681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f669b-0e3c-4c27-bc6e-b900da138e46 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:24.325927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f669b-0e3c-4c27-bc6e-b900da138e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:24.335484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f669b-0e3c-4c27-bc6e-b900da138e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:24.337283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f669b-0e3c-4c27-bc6e-b900da138e46 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:24.346836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 681/1368] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:26.601946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4488dc-493a-4b84-b0ab-594ed35a2400 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:26.602512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4488dc-493a-4b84-b0ab-594ed35a2400 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:26.602512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4488dc-493a-4b84-b0ab-594ed35a2400 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:26.603892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 688/1369] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:26.609630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a07623-6cf5-4b54-bc52-51bc14700ae7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:26.610137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a07623-6cf5-4b54-bc52-51bc14700ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:26.610137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a07623-6cf5-4b54-bc52-51bc14700ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:26.630312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-17a07623-6cf5-4b54-bc52-51bc14700ae7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:26.631635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-17a07623-6cf5-4b54-bc52-51bc14700ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:26.640651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 682/1370] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:32:26.689326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4488dc-493a-4b84-b0ab-594ed35a2400 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:26.704264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4488dc-493a-4b84-b0ab-594ed35a2400 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:26.709607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f0a83-934c-476f-80d1-8fb7548c2fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:26.710255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f0a83-934c-476f-80d1-8fb7548c2fbc service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:26.710532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f0a83-934c-476f-80d1-8fb7548c2fbc service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:26.710834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f0a83-934c-476f-80d1-8fb7548c2fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:26.719916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f0a83-934c-476f-80d1-8fb7548c2fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:26.721512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f0a83-934c-476f-80d1-8fb7548c2fbc service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:26.731874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 689/1371] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:26.863941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3ebc-1b16-4db8-8dc2-00a3952aff8d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:26.864588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3ebc-1b16-4db8-8dc2-00a3952aff8d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:26.864588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf3ebc-1b16-4db8-8dc2-00a3952aff8d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:26.867293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 683/1372] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:26.873632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3b4c-2b43-4648-b25d-33f0ddc8b265 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:26.873925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3b4c-2b43-4648-b25d-33f0ddc8b265 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:26.874168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e3b4c-2b43-4648-b25d-33f0ddc8b265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:26.894741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f3e3b4c-2b43-4648-b25d-33f0ddc8b265 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:26.896036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f3e3b4c-2b43-4648-b25d-33f0ddc8b265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:26.904795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 690/1373] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:27.217287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51834b-e5c8-490b-a93c-7859c27e38b5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:27.217510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51834b-e5c8-490b-a93c-7859c27e38b5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:27.217791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51834b-e5c8-490b-a93c-7859c27e38b5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:27.219754 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 684/1374] 10.222.0.32 () {54 vars in 963 bytes} [Wed May 25 14:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:27.226791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3258245-dcf1-4b3a-af7c-fc8fa8db6a11 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:27.226791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3258245-dcf1-4b3a-af7c-fc8fa8db6a11 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:27.226791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3258245-dcf1-4b3a-af7c-fc8fa8db6a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:27.234834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:32:27.284894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d3258245-dcf1-4b3a-af7c-fc8fa8db6a11 None None] MFA Rules not processed for user `cc3c334dc46b47a3b8e415e913c29249`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:27.287047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3258245-dcf1-4b3a-af7c-fc8fa8db6a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:27.360165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 691/1375] 10.222.0.32 () {58 vars in 1060 bytes} [Wed May 25 14:32:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:32:27.367797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51834b-e5c8-490b-a93c-7859c27e38b5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:27.379288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51834b-e5c8-490b-a93c-7859c27e38b5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:27.383362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572fc05-d1d7-4f4c-9bce-fd9547e53ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:27.384008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572fc05-d1d7-4f4c-9bce-fd9547e53ecc service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:27.384280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572fc05-d1d7-4f4c-9bce-fd9547e53ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:27.384547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572fc05-d1d7-4f4c-9bce-fd9547e53ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:27.394244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572fc05-d1d7-4f4c-9bce-fd9547e53ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:27.395732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572fc05-d1d7-4f4c-9bce-fd9547e53ecc service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:27.405602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 685/1376] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:28.967687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be627-4c00-49a6-8a6c-e7a0ac27b66c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:28.968310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be627-4c00-49a6-8a6c-e7a0ac27b66c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:28.968310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4be627-4c00-49a6-8a6c-e7a0ac27b66c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:28.969620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 692/1377] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:28.975486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf72f8-020c-4f2b-8bc4-a88f3efcd7ff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:28.975970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf72f8-020c-4f2b-8bc4-a88f3efcd7ff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:28.976061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf72f8-020c-4f2b-8bc4-a88f3efcd7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:28.997206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a2cf72f8-020c-4f2b-8bc4-a88f3efcd7ff None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:28.998455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a2cf72f8-020c-4f2b-8bc4-a88f3efcd7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:29.006774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 686/1378] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:29.051981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be627-4c00-49a6-8a6c-e7a0ac27b66c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:29.067375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4be627-4c00-49a6-8a6c-e7a0ac27b66c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:29.073357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694d43d-303a-49cf-bc21-e48dddca098f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:29.074027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694d43d-303a-49cf-bc21-e48dddca098f service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:29.074304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694d43d-303a-49cf-bc21-e48dddca098f service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:29.074570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694d43d-303a-49cf-bc21-e48dddca098f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:29.084100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694d43d-303a-49cf-bc21-e48dddca098f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:29.085761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694d43d-303a-49cf-bc21-e48dddca098f service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:29.096416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 693/1379] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:31.326145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213abb15-a36e-4385-b691-5319b9219cb1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:31.326656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213abb15-a36e-4385-b691-5319b9219cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:31.326656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213abb15-a36e-4385-b691-5319b9219cb1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:31.328055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 687/1380] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:31.333707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f60dc-741f-4705-9f40-771f7edfefb5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:31.334264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f60dc-741f-4705-9f40-771f7edfefb5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:31.334264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f60dc-741f-4705-9f40-771f7edfefb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:31.356182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d2f60dc-741f-4705-9f40-771f7edfefb5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:31.357538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d2f60dc-741f-4705-9f40-771f7edfefb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:31.367012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 694/1381] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:31.410517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213abb15-a36e-4385-b691-5319b9219cb1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:31.426410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213abb15-a36e-4385-b691-5319b9219cb1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:31.433114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f05cf8-f98e-4dd5-99a0-aab0bd992337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:31.434259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05cf8-f98e-4dd5-99a0-aab0bd992337 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:31.434890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05cf8-f98e-4dd5-99a0-aab0bd992337 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:31.435372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05cf8-f98e-4dd5-99a0-aab0bd992337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:31.444772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f05cf8-f98e-4dd5-99a0-aab0bd992337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:31.446610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f05cf8-f98e-4dd5-99a0-aab0bd992337 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:31.457127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 688/1382] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:33.670920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bfdd-ba3d-4fda-ba89-7f80bb48a2a8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:33.671434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bfdd-ba3d-4fda-ba89-7f80bb48a2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:33.671434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bfdd-ba3d-4fda-ba89-7f80bb48a2a8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:33.672948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 695/1383] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:33.678643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce208a-0ab6-4d20-8286-d9b41f61c06b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:33.679138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce208a-0ab6-4d20-8286-d9b41f61c06b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:33.679232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce208a-0ab6-4d20-8286-d9b41f61c06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:33.697986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-acce208a-0ab6-4d20-8286-d9b41f61c06b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:33.699221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-acce208a-0ab6-4d20-8286-d9b41f61c06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:33.707894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 689/1384] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:33.749110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bfdd-ba3d-4fda-ba89-7f80bb48a2a8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:33.763100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bfdd-ba3d-4fda-ba89-7f80bb48a2a8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:33.768430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a3fd7-9f49-49e7-99f1-fcfce473715e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:33.769063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3fd7-9f49-49e7-99f1-fcfce473715e service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:33.769379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3fd7-9f49-49e7-99f1-fcfce473715e service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:33.769605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3fd7-9f49-49e7-99f1-fcfce473715e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:33.778736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a3fd7-9f49-49e7-99f1-fcfce473715e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:33.783020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a3fd7-9f49-49e7-99f1-fcfce473715e service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:33.792754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 696/1385] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:36.281914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52c0-7df2-4b23-ae7b-253d0887f763 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:36.282479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52c0-7df2-4b23-ae7b-253d0887f763 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:36.282774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52c0-7df2-4b23-ae7b-253d0887f763 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:36.284862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 690/1386] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:36.294774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985499-a1f7-477a-8513-8f47650c4b16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:36.295294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985499-a1f7-477a-8513-8f47650c4b16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:36.295617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99985499-a1f7-477a-8513-8f47650c4b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:36.317614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-99985499-a1f7-477a-8513-8f47650c4b16 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:36.319445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-99985499-a1f7-477a-8513-8f47650c4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:36.330242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 697/1387] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:36.384016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52c0-7df2-4b23-ae7b-253d0887f763 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:36.399566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52c0-7df2-4b23-ae7b-253d0887f763 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:36.405440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c075181-0514-4a0e-abd9-3af6d6921bef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:36.406100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c075181-0514-4a0e-abd9-3af6d6921bef service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:36.406431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c075181-0514-4a0e-abd9-3af6d6921bef service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:36.406644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c075181-0514-4a0e-abd9-3af6d6921bef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:36.416124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c075181-0514-4a0e-abd9-3af6d6921bef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:36.417726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c075181-0514-4a0e-abd9-3af6d6921bef service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:36.428094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 691/1388] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:38.662095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b3b4a-0770-4e78-8cf0-8a70cf64b993 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:38.662723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b3b4a-0770-4e78-8cf0-8a70cf64b993 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:38.662723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b3b4a-0770-4e78-8cf0-8a70cf64b993 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:38.664119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 698/1389] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:38.670722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77f525-e297-4c10-b9db-451c58042bb0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:38.671193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77f525-e297-4c10-b9db-451c58042bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:38.671313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae77f525-e297-4c10-b9db-451c58042bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:38.690680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ae77f525-e297-4c10-b9db-451c58042bb0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:38.691952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ae77f525-e297-4c10-b9db-451c58042bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:38.700685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 692/1390] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:38.745007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b3b4a-0770-4e78-8cf0-8a70cf64b993 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:38.759792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59b3b4a-0770-4e78-8cf0-8a70cf64b993 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:38.765474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54333c9e-76f0-41f7-ae0d-9dccf1eb3770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:38.766133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54333c9e-76f0-41f7-ae0d-9dccf1eb3770 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:38.766454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54333c9e-76f0-41f7-ae0d-9dccf1eb3770 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:38.766734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54333c9e-76f0-41f7-ae0d-9dccf1eb3770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:38.779275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54333c9e-76f0-41f7-ae0d-9dccf1eb3770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:38.780877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54333c9e-76f0-41f7-ae0d-9dccf1eb3770 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:38.793243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 699/1391] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:41.030642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416b26-4250-4edd-8ecc-6befa92fa7a5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:41.031238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416b26-4250-4edd-8ecc-6befa92fa7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:41.031376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5416b26-4250-4edd-8ecc-6befa92fa7a5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:41.033308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 693/1392] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:41.039313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a60544-ff14-4dad-a11a-803e9a25067c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:41.039771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a60544-ff14-4dad-a11a-803e9a25067c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:41.039877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a60544-ff14-4dad-a11a-803e9a25067c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:41.058112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-29a60544-ff14-4dad-a11a-803e9a25067c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:41.059367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-29a60544-ff14-4dad-a11a-803e9a25067c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:41.067287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 700/1393] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:41.113749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5416b26-4250-4edd-8ecc-6befa92fa7a5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:41.129665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5416b26-4250-4edd-8ecc-6befa92fa7a5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:41.135474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0394850-7cf8-49bd-935b-76d385df5746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:41.136127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394850-7cf8-49bd-935b-76d385df5746 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:41.136408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394850-7cf8-49bd-935b-76d385df5746 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:41.136691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394850-7cf8-49bd-935b-76d385df5746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:41.145268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0394850-7cf8-49bd-935b-76d385df5746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:41.146955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0394850-7cf8-49bd-935b-76d385df5746 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:41.156636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 694/1394] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:43.372203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fd805-294a-4130-b7ba-16665a4a32a5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.372793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fd805-294a-4130-b7ba-16665a4a32a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.372793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fd805-294a-4130-b7ba-16665a4a32a5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.374714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 701/1395] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:43.380419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e8717-95a0-4e5a-85a4-77835b770332 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.380931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e8717-95a0-4e5a-85a4-77835b770332 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.380931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e8717-95a0-4e5a-85a4-77835b770332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.402501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b56e8717-95a0-4e5a-85a4-77835b770332 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:43.404150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b56e8717-95a0-4e5a-85a4-77835b770332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:43.413534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 695/1396] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:43.457814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fd805-294a-4130-b7ba-16665a4a32a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:43.474128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fd805-294a-4130-b7ba-16665a4a32a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:43.481824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07696e4-4ed5-40f5-bbb8-d52866b80fe9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:43.482185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792449c1-4ae6-4976-8714-394be7083a70 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.482464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792449c1-4ae6-4976-8714-394be7083a70 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.482576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07696e4-4ed5-40f5-bbb8-d52866b80fe9 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.482740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792449c1-4ae6-4976-8714-394be7083a70 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.483602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07696e4-4ed5-40f5-bbb8-d52866b80fe9 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.483602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07696e4-4ed5-40f5-bbb8-d52866b80fe9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.484149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 696/1397] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:43.489057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888be4f3-09f5-40b7-9a05-48d9d7a5319c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.489603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888be4f3-09f5-40b7-9a05-48d9d7a5319c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.489936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888be4f3-09f5-40b7-9a05-48d9d7a5319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.492715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07696e4-4ed5-40f5-bbb8-d52866b80fe9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:43.494311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07696e4-4ed5-40f5-bbb8-d52866b80fe9 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:43.496394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=55 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:32:43.506954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 702/1398] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:43.510671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-888be4f3-09f5-40b7-9a05-48d9d7a5319c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:43.512092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-888be4f3-09f5-40b7-9a05-48d9d7a5319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:43.521210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 697/1399] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:32:43.580072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297863f5-47a3-444a-aa6e-dd14544216f4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.580359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297863f5-47a3-444a-aa6e-dd14544216f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.580805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297863f5-47a3-444a-aa6e-dd14544216f4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.582362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 703/1400] 10.222.0.32 () {54 vars in 962 bytes} [Wed May 25 14:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:43.587805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c99af0-18a2-4026-832c-762d9feb0ff7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.588343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c99af0-18a2-4026-832c-762d9feb0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.588746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c99af0-18a2-4026-832c-762d9feb0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.647134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e0c99af0-18a2-4026-832c-762d9feb0ff7 None None] MFA Rules not processed for user `175dd1ccd6104b18aea8e69bb5ec036a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:43.648548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0c99af0-18a2-4026-832c-762d9feb0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:43.717866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 698/1401] 10.222.0.32 () {58 vars in 1059 bytes} [Wed May 25 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:43.726171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297863f5-47a3-444a-aa6e-dd14544216f4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:43.738402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297863f5-47a3-444a-aa6e-dd14544216f4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:43.742585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9daa344-7d4d-414f-b8f7-0c2baf98f618 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:43.743412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa344-7d4d-414f-b8f7-0c2baf98f618 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:43.743805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa344-7d4d-414f-b8f7-0c2baf98f618 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:43.744093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9daa344-7d4d-414f-b8f7-0c2baf98f618 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:43.754364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daa344-7d4d-414f-b8f7-0c2baf98f618 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:43.755985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9daa344-7d4d-414f-b8f7-0c2baf98f618 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:43.767929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 704/1402] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:44.339699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b544913-119a-4c8e-b7a1-adc24d840c9a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:44.340132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b544913-119a-4c8e-b7a1-adc24d840c9a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:44.340746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b544913-119a-4c8e-b7a1-adc24d840c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:44.417860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7b544913-119a-4c8e-b7a1-adc24d840c9a None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:44.420036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7b544913-119a-4c8e-b7a1-adc24d840c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:44.508909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 699/1403] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:44.547856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297190eb-09ac-4980-8d1d-39f67af08e99 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:44.548406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297190eb-09ac-4980-8d1d-39f67af08e99 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:44.548649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297190eb-09ac-4980-8d1d-39f67af08e99 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:44.550421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 705/1404] 10.222.0.32 () {54 vars in 942 bytes} [Wed May 25 14:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:44.557938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8080c1-0d95-4ea3-b2b1-f7f834ce585e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:44.558298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8080c1-0d95-4ea3-b2b1-f7f834ce585e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:44.558837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8080c1-0d95-4ea3-b2b1-f7f834ce585e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:44.622550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4c8080c1-0d95-4ea3-b2b1-f7f834ce585e None None] MFA Rules not processed for user `5808b2b191a14c358ec62171be87a01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:44.622550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4c8080c1-0d95-4ea3-b2b1-f7f834ce585e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:44.635414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:32:44.709011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 700/1405] 10.222.0.32 () {58 vars in 1039 bytes} [Wed May 25 14:32:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:44.718124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297190eb-09ac-4980-8d1d-39f67af08e99 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:44.736055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297190eb-09ac-4980-8d1d-39f67af08e99 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:44.742647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c918a9-b226-4e37-a379-7dde9b324a92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:44.743353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c918a9-b226-4e37-a379-7dde9b324a92 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:44.743634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c918a9-b226-4e37-a379-7dde9b324a92 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:44.743906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c918a9-b226-4e37-a379-7dde9b324a92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:44.754186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c918a9-b226-4e37-a379-7dde9b324a92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:44.756174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c918a9-b226-4e37-a379-7dde9b324a92 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:44.765025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 706/1406] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:32:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:32:45.763842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d5a62-f556-466d-91f8-ad3697a050aa None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:45.764401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d5a62-f556-466d-91f8-ad3697a050aa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:45.764401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d5a62-f556-466d-91f8-ad3697a050aa None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:45.765777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 701/1407] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:32:45.773279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411e614-0ad6-4a66-9647-b4f62f8405ef None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:45.774276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411e614-0ad6-4a66-9647-b4f62f8405ef None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:45.775161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411e614-0ad6-4a66-9647-b4f62f8405ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:45.802722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a411e614-0ad6-4a66-9647-b4f62f8405ef None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:32:45.804341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a411e614-0ad6-4a66-9647-b4f62f8405ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:32:45.823153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 707/1408] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:32:45.861026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d5a62-f556-466d-91f8-ad3697a050aa None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:32:45.879727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d5a62-f556-466d-91f8-ad3697a050aa None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:32:45.886895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cf757-afe1-4b37-b121-0fc108c0564e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:32:45.887645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cf757-afe1-4b37-b121-0fc108c0564e service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:32:45.887982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cf757-afe1-4b37-b121-0fc108c0564e service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:32:45.888324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cf757-afe1-4b37-b121-0fc108c0564e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:32:45.900319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cf757-afe1-4b37-b121-0fc108c0564e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:32:45.902010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cf757-afe1-4b37-b121-0fc108c0564e service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:32:45.915494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 702/1409] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:32:55.791075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:32:55.911843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=30 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:33:08.084644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a157a4-4a31-4425-82c9-135caca0ab58 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:08.085293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a157a4-4a31-4425-82c9-135caca0ab58 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:08.085293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a157a4-4a31-4425-82c9-135caca0ab58 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:08.086488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 708/1410] 10.222.0.32 () {54 vars in 918 bytes} [Wed May 25 14:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:08.092154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308949a9-d21a-46f8-aad6-88a45716481d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:08.092673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308949a9-d21a-46f8-aad6-88a45716481d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:08.092673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308949a9-d21a-46f8-aad6-88a45716481d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:08.111994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-308949a9-d21a-46f8-aad6-88a45716481d None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:08.113998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-308949a9-d21a-46f8-aad6-88a45716481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:08.122594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 703/1411] 10.222.0.32 () {58 vars in 1015 bytes} [Wed May 25 14:33:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:33:08.136993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd4d20-f550-4b6c-a6a4-84ba4f1ab8b1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:08.137275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd4d20-f550-4b6c-a6a4-84ba4f1ab8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:08.137541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd4d20-f550-4b6c-a6a4-84ba4f1ab8b1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:08.138941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 709/1412] 10.222.0.32 () {54 vars in 918 bytes} [Wed May 25 14:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:08.143535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d86471-0a99-4ea1-a063-4fd7b8bc8f95 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:08.143820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d86471-0a99-4ea1-a063-4fd7b8bc8f95 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:08.144092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d86471-0a99-4ea1-a063-4fd7b8bc8f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:08.163784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-33d86471-0a99-4ea1-a063-4fd7b8bc8f95 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:08.164977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-33d86471-0a99-4ea1-a063-4fd7b8bc8f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:08.175717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 704/1413] 10.222.0.32 () {58 vars in 1015 bytes} [Wed May 25 14:33:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:10.839538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9091634-8c7c-4747-a4e1-328fc83fd23a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:10.840096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9091634-8c7c-4747-a4e1-328fc83fd23a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:10.840096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9091634-8c7c-4747-a4e1-328fc83fd23a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:10.841351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:10.847986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be893-d4cf-4fad-81e7-13f4a433d6d3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:10.848707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be893-d4cf-4fad-81e7-13f4a433d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:10.848855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326be893-d4cf-4fad-81e7-13f4a433d6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:10.875313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-326be893-d4cf-4fad-81e7-13f4a433d6d3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:10.876540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-326be893-d4cf-4fad-81e7-13f4a433d6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:10.885857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:10.934552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9091634-8c7c-4747-a4e1-328fc83fd23a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:10.954047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9091634-8c7c-4747-a4e1-328fc83fd23a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:10.960008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aca298-4d40-4e59-aa1b-ecaa40df11a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:10.960659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aca298-4d40-4e59-aa1b-ecaa40df11a9 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:10.960939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aca298-4d40-4e59-aa1b-ecaa40df11a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:10.961218 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aca298-4d40-4e59-aa1b-ecaa40df11a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:10.969547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aca298-4d40-4e59-aa1b-ecaa40df11a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:10.971186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aca298-4d40-4e59-aa1b-ecaa40df11a9 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:10.980905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 711/1416] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:11.992364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fb0d1-2efa-4819-8dc1-3bb5ff00c283 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:11.993243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fb0d1-2efa-4819-8dc1-3bb5ff00c283 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:11.993243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fb0d1-2efa-4819-8dc1-3bb5ff00c283 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:11.995439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:12.002960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4a7b7-7b2a-4178-b90c-5270efc5abe4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:12.003421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4a7b7-7b2a-4178-b90c-5270efc5abe4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:12.003619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4a7b7-7b2a-4178-b90c-5270efc5abe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:12.021982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-01f4a7b7-7b2a-4178-b90c-5270efc5abe4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:12.023243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01f4a7b7-7b2a-4178-b90c-5270efc5abe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:12.031631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:33:12.194500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fb0d1-2efa-4819-8dc1-3bb5ff00c283 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:12.215923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fb0d1-2efa-4819-8dc1-3bb5ff00c283 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:12.222343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf90a6-3551-41e8-94b8-64690d8e750a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:12.223017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf90a6-3551-41e8-94b8-64690d8e750a service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:12.223299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf90a6-3551-41e8-94b8-64690d8e750a service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:12.223571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf90a6-3551-41e8-94b8-64690d8e750a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:12.232054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf90a6-3551-41e8-94b8-64690d8e750a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:12.233648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf90a6-3551-41e8-94b8-64690d8e750a service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:12.243259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 707/1419] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:13.184859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95938a2-e55b-45e5-9aeb-b3f059373be8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:13.185482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95938a2-e55b-45e5-9aeb-b3f059373be8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:13.185482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95938a2-e55b-45e5-9aeb-b3f059373be8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:13.186781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:13.196101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b2a9-f6d6-4e0c-af3f-c644e8cdbe9f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:13.196621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b2a9-f6d6-4e0c-af3f-c644e8cdbe9f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:13.196723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035b2a9-f6d6-4e0c-af3f-c644e8cdbe9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:13.216664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e035b2a9-f6d6-4e0c-af3f-c644e8cdbe9f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:13.217872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e035b2a9-f6d6-4e0c-af3f-c644e8cdbe9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:13.227550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:13.388027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95938a2-e55b-45e5-9aeb-b3f059373be8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:13.412758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95938a2-e55b-45e5-9aeb-b3f059373be8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:13.419984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dd3b1-bec7-4dd9-84a6-64b9bb8cff23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:13.420623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dd3b1-bec7-4dd9-84a6-64b9bb8cff23 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:13.420902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dd3b1-bec7-4dd9-84a6-64b9bb8cff23 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:13.421167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dd3b1-bec7-4dd9-84a6-64b9bb8cff23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:13.430013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dd3b1-bec7-4dd9-84a6-64b9bb8cff23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:13.431722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dd3b1-bec7-4dd9-84a6-64b9bb8cff23 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:13.441431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 714/1422] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:14.350599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd5a2-8697-4361-891a-445ad36047b4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:14.351209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd5a2-8697-4361-891a-445ad36047b4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:14.351209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecd5a2-8697-4361-891a-445ad36047b4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:14.352576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:14.357914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a44c-3180-4fc2-8147-8bc4b468ed2d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:14.358357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a44c-3180-4fc2-8147-8bc4b468ed2d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:14.358450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a44c-3180-4fc2-8147-8bc4b468ed2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:14.380459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f537a44c-3180-4fc2-8147-8bc4b468ed2d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:14.381669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f537a44c-3180-4fc2-8147-8bc4b468ed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:14.391217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:14.543207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd5a2-8697-4361-891a-445ad36047b4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:14.563059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecd5a2-8697-4361-891a-445ad36047b4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:14.569389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce38baf-ee1a-4ea1-b6b0-7c3fe0d37c57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:14.570096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38baf-ee1a-4ea1-b6b0-7c3fe0d37c57 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:14.570417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38baf-ee1a-4ea1-b6b0-7c3fe0d37c57 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:14.570771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce38baf-ee1a-4ea1-b6b0-7c3fe0d37c57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:14.579246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce38baf-ee1a-4ea1-b6b0-7c3fe0d37c57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:14.580838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce38baf-ee1a-4ea1-b6b0-7c3fe0d37c57 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:14.590707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 710/1425] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:15.511094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d024c-a91d-4e93-b899-b647a6e436ec None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:15.511709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d024c-a91d-4e93-b899-b647a6e436ec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:15.511709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d024c-a91d-4e93-b899-b647a6e436ec None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:15.513027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:15.519283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b423733-8269-48b0-ba06-06a18bfffa3c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:15.520045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b423733-8269-48b0-ba06-06a18bfffa3c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:15.520220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b423733-8269-48b0-ba06-06a18bfffa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:15.546883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3b423733-8269-48b0-ba06-06a18bfffa3c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:15.548111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3b423733-8269-48b0-ba06-06a18bfffa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:15.556467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:15.706057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d024c-a91d-4e93-b899-b647a6e436ec None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:15.727377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d024c-a91d-4e93-b899-b647a6e436ec None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:15.734055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f750e-7db4-4b86-8378-c794fcc26df3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:15.734696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f750e-7db4-4b86-8378-c794fcc26df3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:15.735011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f750e-7db4-4b86-8378-c794fcc26df3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:15.735283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f750e-7db4-4b86-8378-c794fcc26df3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:15.744075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f750e-7db4-4b86-8378-c794fcc26df3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:15.745651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f750e-7db4-4b86-8378-c794fcc26df3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:15.755267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 717/1428] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:16.603843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22093ee0-469f-49c2-8a8c-0cf0122ae038 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:16.604625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22093ee0-469f-49c2-8a8c-0cf0122ae038 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:16.604772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22093ee0-469f-49c2-8a8c-0cf0122ae038 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:16.606995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:16.615454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f922-6d21-49fb-be9e-ae87d4b859b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:16.615946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f922-6d21-49fb-be9e-ae87d4b859b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:16.616122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547f922-6d21-49fb-be9e-ae87d4b859b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:16.635973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d547f922-6d21-49fb-be9e-ae87d4b859b8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:16.637182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d547f922-6d21-49fb-be9e-ae87d4b859b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:16.646000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:16.795556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22093ee0-469f-49c2-8a8c-0cf0122ae038 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:16.814324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22093ee0-469f-49c2-8a8c-0cf0122ae038 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:16.820944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43346df2-a61b-4d51-93fe-4975ca517a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:16.821585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43346df2-a61b-4d51-93fe-4975ca517a5f service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:16.821860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43346df2-a61b-4d51-93fe-4975ca517a5f service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:16.822131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43346df2-a61b-4d51-93fe-4975ca517a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:16.830688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43346df2-a61b-4d51-93fe-4975ca517a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:16.832327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43346df2-a61b-4d51-93fe-4975ca517a5f service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:16.842040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 713/1431] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:17.702685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bec5c4-e3b4-4d21-a6d3-4c84d6e812c6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:17.703336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bec5c4-e3b4-4d21-a6d3-4c84d6e812c6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:17.703336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bec5c4-e3b4-4d21-a6d3-4c84d6e812c6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:17.704633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:17.710849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673b4f7-40ac-480f-8a6a-22b8ca5e2caf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:17.711584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673b4f7-40ac-480f-8a6a-22b8ca5e2caf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:17.711764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673b4f7-40ac-480f-8a6a-22b8ca5e2caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:17.736281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b673b4f7-40ac-480f-8a6a-22b8ca5e2caf None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:17.737492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b673b4f7-40ac-480f-8a6a-22b8ca5e2caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:17.748081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:17.902587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bec5c4-e3b4-4d21-a6d3-4c84d6e812c6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:17.923510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bec5c4-e3b4-4d21-a6d3-4c84d6e812c6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:17.932779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e4796-c823-48a7-be6f-ad1519cf1be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:17.933678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e4796-c823-48a7-be6f-ad1519cf1be0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:17.933960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e4796-c823-48a7-be6f-ad1519cf1be0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:17.934228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e4796-c823-48a7-be6f-ad1519cf1be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:17.943795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e4796-c823-48a7-be6f-ad1519cf1be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:17.945525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e4796-c823-48a7-be6f-ad1519cf1be0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:17.955204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 720/1434] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:18.800155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ccd4-1d7c-4cbc-9bd5-4323b8b646fd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:18.800778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ccd4-1d7c-4cbc-9bd5-4323b8b646fd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:18.800965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ccd4-1d7c-4cbc-9bd5-4323b8b646fd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:18.803111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:18.808388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3cbc9-dc3c-4b06-9b02-0f6067faba9a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:18.808839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3cbc9-dc3c-4b06-9b02-0f6067faba9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:18.808959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3cbc9-dc3c-4b06-9b02-0f6067faba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:18.827601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1fc3cbc9-dc3c-4b06-9b02-0f6067faba9a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:18.828796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fc3cbc9-dc3c-4b06-9b02-0f6067faba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:18.837364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:18.991518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7ccd4-1d7c-4cbc-9bd5-4323b8b646fd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:19.010943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7ccd4-1d7c-4cbc-9bd5-4323b8b646fd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:19.016736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1b8fb-3f45-405f-9bcd-174e582bad00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:19.017376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b8fb-3f45-405f-9bcd-174e582bad00 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:19.017655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b8fb-3f45-405f-9bcd-174e582bad00 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:19.017926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b8fb-3f45-405f-9bcd-174e582bad00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:19.026607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b8fb-3f45-405f-9bcd-174e582bad00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:19.028361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b8fb-3f45-405f-9bcd-174e582bad00 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:19.038054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 716/1437] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:19.895551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b591b-62a1-4eeb-ac5f-d1e3737bccbe None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:19.896414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b591b-62a1-4eeb-ac5f-d1e3737bccbe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:19.896414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b591b-62a1-4eeb-ac5f-d1e3737bccbe None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:19.898291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:33:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:19.906123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889aea-2990-4c56-b543-5f663f844b40 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:19.906531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889aea-2990-4c56-b543-5f663f844b40 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:19.906703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6889aea-2990-4c56-b543-5f663f844b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:19.926638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c6889aea-2990-4c56-b543-5f663f844b40 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:19.927862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c6889aea-2990-4c56-b543-5f663f844b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:19.935946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:20.084330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b591b-62a1-4eeb-ac5f-d1e3737bccbe None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:20.106978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b591b-62a1-4eeb-ac5f-d1e3737bccbe None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:20.112873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b11f74-226f-441a-8b8e-e137e7e33b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:20.113533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b11f74-226f-441a-8b8e-e137e7e33b58 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:20.113802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b11f74-226f-441a-8b8e-e137e7e33b58 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:20.114068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b11f74-226f-441a-8b8e-e137e7e33b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:20.122276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b11f74-226f-441a-8b8e-e137e7e33b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:20.123872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b11f74-226f-441a-8b8e-e137e7e33b58 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:20.133352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 723/1440] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:22.187774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d26f69-ce56-43ba-9bc6-380fea4d650d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:22.188345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d26f69-ce56-43ba-9bc6-380fea4d650d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:22.188345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d26f69-ce56-43ba-9bc6-380fea4d650d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:22.189699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:22.196311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06a0da-c09d-4ea5-bd6d-8cc93e1144bb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:22.197091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06a0da-c09d-4ea5-bd6d-8cc93e1144bb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:22.197259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06a0da-c09d-4ea5-bd6d-8cc93e1144bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:22.226393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3a06a0da-c09d-4ea5-bd6d-8cc93e1144bb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:22.227604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a06a0da-c09d-4ea5-bd6d-8cc93e1144bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:22.237558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:22.279945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d26f69-ce56-43ba-9bc6-380fea4d650d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:22.295147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d26f69-ce56-43ba-9bc6-380fea4d650d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:22.298781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae62a95-5aeb-4684-8d49-8e3af9470d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:22.299476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae62a95-5aeb-4684-8d49-8e3af9470d58 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:22.299757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae62a95-5aeb-4684-8d49-8e3af9470d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:22.300031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae62a95-5aeb-4684-8d49-8e3af9470d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:22.309308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae62a95-5aeb-4684-8d49-8e3af9470d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:22.310775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae62a95-5aeb-4684-8d49-8e3af9470d58 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:22.320410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 719/1443] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:24.775625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aee61-05a6-4ccc-8875-538ec31bfb86 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:24.776226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aee61-05a6-4ccc-8875-538ec31bfb86 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:24.776226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48aee61-05a6-4ccc-8875-538ec31bfb86 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:24.777613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:24.783063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faa3bb-5476-4c9c-b1b0-6c53a83e2111 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:24.783534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faa3bb-5476-4c9c-b1b0-6c53a83e2111 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:24.783630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1faa3bb-5476-4c9c-b1b0-6c53a83e2111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:24.802997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f1faa3bb-5476-4c9c-b1b0-6c53a83e2111 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:24.804256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f1faa3bb-5476-4c9c-b1b0-6c53a83e2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:24.812244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:24.850586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aee61-05a6-4ccc-8875-538ec31bfb86 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:24.863801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48aee61-05a6-4ccc-8875-538ec31bfb86 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:24.868414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456267c0-e088-43e7-970c-0f6a2d6619fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:24.869052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456267c0-e088-43e7-970c-0f6a2d6619fd service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:24.869323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456267c0-e088-43e7-970c-0f6a2d6619fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:24.869603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456267c0-e088-43e7-970c-0f6a2d6619fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:24.879277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456267c0-e088-43e7-970c-0f6a2d6619fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:24.880705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456267c0-e088-43e7-970c-0f6a2d6619fd service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:24.890232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 726/1446] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:27.446720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef5b8d-f879-4bed-90aa-ba4fd69ee8ee None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:27.447320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef5b8d-f879-4bed-90aa-ba4fd69ee8ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:27.447320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef5b8d-f879-4bed-90aa-ba4fd69ee8ee None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:27.448687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:27.455243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10ee7-c963-4e95-b50a-391b0c3f9ab7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:27.455933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10ee7-c963-4e95-b50a-391b0c3f9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:27.456146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e10ee7-c963-4e95-b50a-391b0c3f9ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:27.482470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7e10ee7-c963-4e95-b50a-391b0c3f9ab7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:27.483697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7e10ee7-c963-4e95-b50a-391b0c3f9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:27.492399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:27.530124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef5b8d-f879-4bed-90aa-ba4fd69ee8ee None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:27.542296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef5b8d-f879-4bed-90aa-ba4fd69ee8ee None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:27.546098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cabf1-6958-41d7-91f7-1c718f7952b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:27.546736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cabf1-6958-41d7-91f7-1c718f7952b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:27.547051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cabf1-6958-41d7-91f7-1c718f7952b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:27.547302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cabf1-6958-41d7-91f7-1c718f7952b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:27.556375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cabf1-6958-41d7-91f7-1c718f7952b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:27.557821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cabf1-6958-41d7-91f7-1c718f7952b1 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:27.567102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 722/1449] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:29.893548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91261a-35d4-4510-aee0-d12fbad511f0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:29.894188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91261a-35d4-4510-aee0-d12fbad511f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:29.894188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91261a-35d4-4510-aee0-d12fbad511f0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:29.895517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:29.902125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1a639-ff31-45a3-9931-524ea035dae3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:29.902920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1a639-ff31-45a3-9931-524ea035dae3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:29.903081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1a639-ff31-45a3-9931-524ea035dae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:29.926485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f9c1a639-ff31-45a3-9931-524ea035dae3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:29.927745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f9c1a639-ff31-45a3-9931-524ea035dae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:29.937369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:29.979221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91261a-35d4-4510-aee0-d12fbad511f0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:29.991223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91261a-35d4-4510-aee0-d12fbad511f0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:29.995084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dedbcb-aa6b-4534-a62e-1363d4714da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:29.995738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dedbcb-aa6b-4534-a62e-1363d4714da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:29.996013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dedbcb-aa6b-4534-a62e-1363d4714da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:29.996284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dedbcb-aa6b-4534-a62e-1363d4714da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:30.005877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dedbcb-aa6b-4534-a62e-1363d4714da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:30.007409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dedbcb-aa6b-4534-a62e-1363d4714da7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:30.016750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 729/1452] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:32.259761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b838df8-37d9-4e3d-afcd-199f0f3007b7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:32.260743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b838df8-37d9-4e3d-afcd-199f0f3007b7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:32.260743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b838df8-37d9-4e3d-afcd-199f0f3007b7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:32.263370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:32.271701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ba975-808e-40ba-986c-62f4540e0ac0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:32.272169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ba975-808e-40ba-986c-62f4540e0ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:32.272280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ba975-808e-40ba-986c-62f4540e0ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:32.290780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ad1ba975-808e-40ba-986c-62f4540e0ac0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:32.292057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad1ba975-808e-40ba-986c-62f4540e0ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:32.300374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:33:32.340040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b838df8-37d9-4e3d-afcd-199f0f3007b7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:32.354498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b838df8-37d9-4e3d-afcd-199f0f3007b7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:32.361543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e2b34-a073-4ae9-bd69-dc99a02ce66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:32.362859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e2b34-a073-4ae9-bd69-dc99a02ce66e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:32.363223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e2b34-a073-4ae9-bd69-dc99a02ce66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:32.363577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e2b34-a073-4ae9-bd69-dc99a02ce66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:32.374862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e2b34-a073-4ae9-bd69-dc99a02ce66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:32.376560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e2b34-a073-4ae9-bd69-dc99a02ce66e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:32.386675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 725/1455] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:34.628848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f163e-3fd3-4668-82f3-d81f094678d5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:34.629371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f163e-3fd3-4668-82f3-d81f094678d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:34.629371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f163e-3fd3-4668-82f3-d81f094678d5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:34.630798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:34.636402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5abb13-8600-42e3-b634-9a95a7c749f6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:34.637329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5abb13-8600-42e3-b634-9a95a7c749f6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:34.637816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5abb13-8600-42e3-b634-9a95a7c749f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:34.657126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5c5abb13-8600-42e3-b634-9a95a7c749f6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:34.658325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c5abb13-8600-42e3-b634-9a95a7c749f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:34.666582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:34.704427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f163e-3fd3-4668-82f3-d81f094678d5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:34.717200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f163e-3fd3-4668-82f3-d81f094678d5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:34.721413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41f07-509e-4d75-90e9-a871d0113c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:34.722046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41f07-509e-4d75-90e9-a871d0113c8e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:34.722309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41f07-509e-4d75-90e9-a871d0113c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:34.722572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41f07-509e-4d75-90e9-a871d0113c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:34.731706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41f07-509e-4d75-90e9-a871d0113c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:34.733121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41f07-509e-4d75-90e9-a871d0113c8e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:34.742555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 732/1458] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:36.993266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a6d3-a258-4194-ba51-363e25e17174 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:36.993805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a6d3-a258-4194-ba51-363e25e17174 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:36.993805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a6d3-a258-4194-ba51-363e25e17174 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:36.995783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:37.001436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242ba61-c40a-48fc-be77-ae4a858bf7ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:37.001931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242ba61-c40a-48fc-be77-ae4a858bf7ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:37.001931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242ba61-c40a-48fc-be77-ae4a858bf7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:37.009024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=70 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:33:37.022407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f242ba61-c40a-48fc-be77-ae4a858bf7ca None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:37.023637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f242ba61-c40a-48fc-be77-ae4a858bf7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:37.031574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:37.071932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73a6d3-a258-4194-ba51-363e25e17174 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:37.081248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=73 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:33:37.088017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73a6d3-a258-4194-ba51-363e25e17174 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:37.091826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74abe0-bf50-409d-aae0-ab7b3388292a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:37.092466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74abe0-bf50-409d-aae0-ab7b3388292a service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:37.092742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74abe0-bf50-409d-aae0-ab7b3388292a service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:37.093014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74abe0-bf50-409d-aae0-ab7b3388292a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:37.103117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74abe0-bf50-409d-aae0-ab7b3388292a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:37.104568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74abe0-bf50-409d-aae0-ab7b3388292a service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:37.114586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 728/1461] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:39.359553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6131185-de9d-48f2-a63c-3aedcb314129 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:39.360160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6131185-de9d-48f2-a63c-3aedcb314129 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:39.360160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6131185-de9d-48f2-a63c-3aedcb314129 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:39.361484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:39.367069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429538-c9e8-49ff-9d59-f8373c5fd75c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:39.367521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429538-c9e8-49ff-9d59-f8373c5fd75c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:39.367628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88429538-c9e8-49ff-9d59-f8373c5fd75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:39.388294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-88429538-c9e8-49ff-9d59-f8373c5fd75c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:39.389496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-88429538-c9e8-49ff-9d59-f8373c5fd75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:39.399044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:39.443668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6131185-de9d-48f2-a63c-3aedcb314129 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:39.456328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6131185-de9d-48f2-a63c-3aedcb314129 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:39.460349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cd954-9a67-4784-92db-404b0dae8df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:39.460983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cd954-9a67-4784-92db-404b0dae8df3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:39.461262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cd954-9a67-4784-92db-404b0dae8df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:39.461530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cd954-9a67-4784-92db-404b0dae8df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:39.470996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cd954-9a67-4784-92db-404b0dae8df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:39.472425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cd954-9a67-4784-92db-404b0dae8df3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:39.482001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 735/1464] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:41.669657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a6a84-78cd-4f24-bbeb-43b097bc1d05 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:41.670229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a6a84-78cd-4f24-bbeb-43b097bc1d05 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:41.670229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a6a84-78cd-4f24-bbeb-43b097bc1d05 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:41.671621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:41.678137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c9dae-4101-4ea9-a5c8-4c0e18d99e52 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:41.679112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c9dae-4101-4ea9-a5c8-4c0e18d99e52 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:41.679112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c9dae-4101-4ea9-a5c8-4c0e18d99e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:41.703311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f52c9dae-4101-4ea9-a5c8-4c0e18d99e52 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:41.704523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f52c9dae-4101-4ea9-a5c8-4c0e18d99e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:41.713056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:41.748898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a6a84-78cd-4f24-bbeb-43b097bc1d05 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:41.761386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a6a84-78cd-4f24-bbeb-43b097bc1d05 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:41.765477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1b062-9e9a-4a87-b2ad-f9a27920dea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:41.766114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1b062-9e9a-4a87-b2ad-f9a27920dea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:41.766395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1b062-9e9a-4a87-b2ad-f9a27920dea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:41.766669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1b062-9e9a-4a87-b2ad-f9a27920dea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:41.776466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1b062-9e9a-4a87-b2ad-f9a27920dea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:41.777902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1b062-9e9a-4a87-b2ad-f9a27920dea9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:41.788248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 731/1467] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:43.989503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54896807-7bcc-4e02-953f-17d1e852ca06 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:43.990110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54896807-7bcc-4e02-953f-17d1e852ca06 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:43.990110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54896807-7bcc-4e02-953f-17d1e852ca06 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:43.991503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:33:43.996899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae57d3-5959-4e63-8e55-af5518731aa3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:43.997357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae57d3-5959-4e63-8e55-af5518731aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:43.997454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae57d3-5959-4e63-8e55-af5518731aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:44.016030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b0ae57d3-5959-4e63-8e55-af5518731aa3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:33:44.017224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b0ae57d3-5959-4e63-8e55-af5518731aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:33:44.025907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:33:44.064485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54896807-7bcc-4e02-953f-17d1e852ca06 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:33:44.076959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54896807-7bcc-4e02-953f-17d1e852ca06 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:33:44.081281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69745815-1744-42ea-a93c-d1d9c2863236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:33:44.081918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745815-1744-42ea-a93c-d1d9c2863236 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:33:44.082152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745815-1744-42ea-a93c-d1d9c2863236 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:33:44.082455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69745815-1744-42ea-a93c-d1d9c2863236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:33:44.092286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745815-1744-42ea-a93c-d1d9c2863236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:33:44.093703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69745815-1744-42ea-a93c-d1d9c2863236 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:33:44.103612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 738/1470] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:33:54.010868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=17 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:33:54.098945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:16.807213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9b95-b00f-484a-bcdf-7f3a2014758a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.807709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9b95-b00f-484a-bcdf-7f3a2014758a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.807816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9b95-b00f-484a-bcdf-7f3a2014758a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.809151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 733/1471] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:16.814682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcba1b9-0004-4e26-92c8-a423e8b99ff8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.815112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcba1b9-0004-4e26-92c8-a423e8b99ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.815283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcba1b9-0004-4e26-92c8-a423e8b99ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.834375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bbcba1b9-0004-4e26-92c8-a423e8b99ff8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:16.835705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bbcba1b9-0004-4e26-92c8-a423e8b99ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:16.844971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 739/1472] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:34:16.893923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5122-7155-4181-aa58-0bf10e01b650 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.894204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5122-7155-4181-aa58-0bf10e01b650 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.894472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a5122-7155-4181-aa58-0bf10e01b650 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.895900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 734/1473] 10.222.0.32 () {54 vars in 962 bytes} [Wed May 25 14:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:16.900924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ab83-5e92-417d-b5de-3364f0d97267 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.901207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ab83-5e92-417d-b5de-3364f0d97267 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.901478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c611ab83-5e92-417d-b5de-3364f0d97267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.920046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c611ab83-5e92-417d-b5de-3364f0d97267 None None] MFA Rules not processed for user `175dd1ccd6104b18aea8e69bb5ec036a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:16.921231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c611ab83-5e92-417d-b5de-3364f0d97267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:16.927873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 740/1474] 10.222.0.32 () {58 vars in 1059 bytes} [Wed May 25 14:34:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:34:16.935292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5122-7155-4181-aa58-0bf10e01b650 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:16.947189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a5122-7155-4181-aa58-0bf10e01b650 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:16.951333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792e83-a7d7-4aa2-b90e-6eff21539714 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:16.951961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792e83-a7d7-4aa2-b90e-6eff21539714 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:16.952241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792e83-a7d7-4aa2-b90e-6eff21539714 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:16.952507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792e83-a7d7-4aa2-b90e-6eff21539714 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:16.961572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a792e83-a7d7-4aa2-b90e-6eff21539714 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:16.963031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a792e83-a7d7-4aa2-b90e-6eff21539714 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:16.972365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 735/1475] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:34:19.269206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beef401-822e-4dc1-8129-5dc57825a343 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:19.270083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beef401-822e-4dc1-8129-5dc57825a343 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:19.270083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beef401-822e-4dc1-8129-5dc57825a343 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:19.272341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 741/1476] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:19.277824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76042c-f07b-48e3-a9b0-1f1082d6f968 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:19.278269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76042c-f07b-48e3-a9b0-1f1082d6f968 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:19.278378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76042c-f07b-48e3-a9b0-1f1082d6f968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:19.298249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1c76042c-f07b-48e3-a9b0-1f1082d6f968 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:19.299484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c76042c-f07b-48e3-a9b0-1f1082d6f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:19.308850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 736/1477] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:34:19.337862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beef401-822e-4dc1-8129-5dc57825a343 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:19.352210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beef401-822e-4dc1-8129-5dc57825a343 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:19.357319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:19.357966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:19.358243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:19.358510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:19.359274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:19.361097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:34:19.361097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:34:19.361097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:34:19.362234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa0039-4677-4f6a-af8a-5a880b9b5c19 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:19.381455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 742/1478] 10.222.0.32 () {56 vars in 1108 bytes} [Wed May 25 14:34:19 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:34:21.580048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc30446-25fa-4ff5-aa1b-4d6a7e053aad None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.580606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc30446-25fa-4ff5-aa1b-4d6a7e053aad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.580606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc30446-25fa-4ff5-aa1b-4d6a7e053aad None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.582790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 737/1479] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:21.589837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b083b2-c397-4223-9ca5-608641ed195f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.590272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b083b2-c397-4223-9ca5-608641ed195f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.590363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b083b2-c397-4223-9ca5-608641ed195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.609283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c8b083b2-c397-4223-9ca5-608641ed195f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:21.610511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8b083b2-c397-4223-9ca5-608641ed195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:21.620330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 743/1480] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:34:21.664804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc30446-25fa-4ff5-aa1b-4d6a7e053aad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:21.683805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc30446-25fa-4ff5-aa1b-4d6a7e053aad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:21.689445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7896-d19c-4663-a94b-23615512ea86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:21.690086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7896-d19c-4663-a94b-23615512ea86 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:21.690361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7896-d19c-4663-a94b-23615512ea86 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:21.690629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7896-d19c-4663-a94b-23615512ea86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:21.700418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7896-d19c-4663-a94b-23615512ea86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:21.702283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7896-d19c-4663-a94b-23615512ea86 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:21.712665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 738/1481] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:34:24.114351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f79a54-0d79-455a-90dc-c934ae0bf2e8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:24.114976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f79a54-0d79-455a-90dc-c934ae0bf2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:24.114976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f79a54-0d79-455a-90dc-c934ae0bf2e8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:24.116317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 744/1482] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:24.122905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad468f95-dd01-4a27-88f8-a4dcd2fb02e5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:24.123570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad468f95-dd01-4a27-88f8-a4dcd2fb02e5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:24.123800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad468f95-dd01-4a27-88f8-a4dcd2fb02e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:24.146536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ad468f95-dd01-4a27-88f8-a4dcd2fb02e5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:24.147860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ad468f95-dd01-4a27-88f8-a4dcd2fb02e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:24.156944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 739/1483] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:34:24.192502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f79a54-0d79-455a-90dc-c934ae0bf2e8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:24.203966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f79a54-0d79-455a-90dc-c934ae0bf2e8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:24.207974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74922f-439a-4ce7-8e87-6b3c864adcef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:24.208605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74922f-439a-4ce7-8e87-6b3c864adcef service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:24.208855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74922f-439a-4ce7-8e87-6b3c864adcef service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:24.209124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74922f-439a-4ce7-8e87-6b3c864adcef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:24.220023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74922f-439a-4ce7-8e87-6b3c864adcef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:24.221439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74922f-439a-4ce7-8e87-6b3c864adcef service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:24.231093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 745/1484] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:34:26.461911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d0ec7-e443-4f39-9453-841e66b94e3f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.462517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d0ec7-e443-4f39-9453-841e66b94e3f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.462517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d0ec7-e443-4f39-9453-841e66b94e3f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.463890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 740/1485] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:26.470185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b141ed-4d34-49fa-9e6d-4f24a9750e1a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.470635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b141ed-4d34-49fa-9e6d-4f24a9750e1a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.470744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b141ed-4d34-49fa-9e6d-4f24a9750e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.491710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-41b141ed-4d34-49fa-9e6d-4f24a9750e1a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:26.493063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-41b141ed-4d34-49fa-9e6d-4f24a9750e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:26.502369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 746/1486] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:34:26.542738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d0ec7-e443-4f39-9453-841e66b94e3f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:26.562204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d0ec7-e443-4f39-9453-841e66b94e3f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:26.567262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c3eb-855e-4fd8-910f-0b3e185ea32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:26.567898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c3eb-855e-4fd8-910f-0b3e185ea32e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:26.568178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c3eb-855e-4fd8-910f-0b3e185ea32e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:26.568445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c3eb-855e-4fd8-910f-0b3e185ea32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:26.577851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c3eb-855e-4fd8-910f-0b3e185ea32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:26.579329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c3eb-855e-4fd8-910f-0b3e185ea32e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:26.589192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 741/1487] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:34:28.983581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a930-667c-4efa-a97b-3a73f621f4c8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:28.984210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a930-667c-4efa-a97b-3a73f621f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:28.984210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6a930-667c-4efa-a97b-3a73f621f4c8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:28.985552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 747/1488] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:28.992862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de51d9-1466-47fd-a07b-d881e6078eb3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:28.993675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de51d9-1466-47fd-a07b-d881e6078eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:28.993675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de51d9-1466-47fd-a07b-d881e6078eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.014281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-51de51d9-1466-47fd-a07b-d881e6078eb3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:29.015529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-51de51d9-1466-47fd-a07b-d881e6078eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:29.024785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 742/1489] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:34:29.072783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a930-667c-4efa-a97b-3a73f621f4c8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:29.091093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6a930-667c-4efa-a97b-3a73f621f4c8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:29.095500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63bb91-e3ce-4784-929f-03a6ff454321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:29.096142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63bb91-e3ce-4784-929f-03a6ff454321 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:29.096421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63bb91-e3ce-4784-929f-03a6ff454321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:29.096692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63bb91-e3ce-4784-929f-03a6ff454321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:29.106234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63bb91-e3ce-4784-929f-03a6ff454321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:29.107696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63bb91-e3ce-4784-929f-03a6ff454321 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:29.117632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 748/1490] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:34:31.517029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aee6-d618-4473-bb10-05f7e0960701 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:31.518033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aee6-d618-4473-bb10-05f7e0960701 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:31.518033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3aee6-d618-4473-bb10-05f7e0960701 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:31.520206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 743/1491] 10.222.0.32 () {54 vars in 926 bytes} [Wed May 25 14:34:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:34:31.528336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e0b90-fbf5-4993-b261-d644db7e626b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:31.528805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e0b90-fbf5-4993-b261-d644db7e626b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:31.528911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e0b90-fbf5-4993-b261-d644db7e626b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:31.553638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0f2e0b90-fbf5-4993-b261-d644db7e626b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:34:31.554836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f2e0b90-fbf5-4993-b261-d644db7e626b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:34:31.564056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 749/1492] 10.222.0.32 () {58 vars in 1023 bytes} [Wed May 25 14:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:34:31.604928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aee6-d618-4473-bb10-05f7e0960701 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:34:31.618519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3aee6-d618-4473-bb10-05f7e0960701 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:34:31.624148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254cac6-ddd8-490c-9c87-b07dd4d89175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:34:31.625004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254cac6-ddd8-490c-9c87-b07dd4d89175 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:34:31.625286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254cac6-ddd8-490c-9c87-b07dd4d89175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:34:31.625527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254cac6-ddd8-490c-9c87-b07dd4d89175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:34:31.636544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254cac6-ddd8-490c-9c87-b07dd4d89175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:34:31.638071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254cac6-ddd8-490c-9c87-b07dd4d89175 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:34:31.648511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 744/1493] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:34:41.547145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:34:41.643601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:11.634081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.634698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.634698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.659306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:11.660513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:11.669272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 750/1494] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:36:11.679065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.679551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.679658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.699801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:11.700989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:11.710557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 745/1495] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:36:11.715825 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.725936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85190a3c-8156-4af4-ac4b-71339fe0fa6d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.731010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.731666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.731941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.732248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.732977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:36:11.733281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.734303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:11.734303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:36:11.734303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:36:11.735254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.747118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 751/1496] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:36:11.755232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.764814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588bda7-f460-4ad6-8bf3-3a78aa17c64a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.769922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.770568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.770881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.771151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.771893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.773782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:11.773782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:36:11.773782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:36:11.775030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.819710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 746/1497] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:36:11 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:11.824416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.834606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ea5b1-96b5-4064-80af-c50d8efe8e4b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.898409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.899190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.899518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.899874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.900973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.902447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:11.902447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:36:11.902447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:36:11.903673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.926129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 752/1498] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:36:11 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:36:11.931716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.943225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75978901-fc68-4395-b9c9-ec5ee34de5c3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.949600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.950285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.950563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.950946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.951672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.952685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:11.952685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:36:11.952685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:36:11.953564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:11.966617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 747/1499] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:36:11.970729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:11.981341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9bb606-d559-4250-95ef-69d1a48c896e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:11.986487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:11.987210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:11.987494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:11.987761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:11.988658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:11.989688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:11.989688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:36:11.989688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:36:11.990554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13c339-c9da-4ef2-b261-fd33b3470156 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.005286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 753/1500] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:36:12.009821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.020492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb13e92-1a20-4638-8245-532f4726428a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.025609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.026489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.026793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.027123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] PATH_INFO: `/v3/projects/3159825930c64d0cb6f2b3823b2536c2/users/53f8633f932b4100bdabacb39e84eb4c/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.043368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3159825930c64d0cb6f2b3823b2536c2, user_id=53f8633f932b4100bdabacb39e84eb4c, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.044406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:12.044406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:36:12.044406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:36:12.046333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7fa15-876b-4fb8-bb90-8e3f772799e7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.063117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 748/1501] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:36:12 2022] PUT /identity/v3/projects/3159825930c64d0cb6f2b3823b2536c2/users/53f8633f932b4100bdabacb39e84eb4c/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:12.078649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36d86d-7335-420c-b469-68fb1c4120a9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.078957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36d86d-7335-420c-b469-68fb1c4120a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.079227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36d86d-7335-420c-b469-68fb1c4120a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.105747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ce36d86d-7335-420c-b469-68fb1c4120a9 None None] MFA Rules not processed for user `53f8633f932b4100bdabacb39e84eb4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:12.106966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce36d86d-7335-420c-b469-68fb1c4120a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:12.172078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 754/1502] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:36:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:36:12.190469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acea1b-f957-499b-81b2-fae68945f496 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.190808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acea1b-f957-499b-81b2-fae68945f496 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.191225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acea1b-f957-499b-81b2-fae68945f496 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.193189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 749/1503] 10.222.0.32 () {54 vars in 963 bytes} [Wed May 25 14:36:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:12.200026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d58e-c036-475b-8f97-b19aad6a2ec5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.200316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d58e-c036-475b-8f97-b19aad6a2ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.200591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22d58e-c036-475b-8f97-b19aad6a2ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.219899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d22d58e-c036-475b-8f97-b19aad6a2ec5 None None] MFA Rules not processed for user `cc3c334dc46b47a3b8e415e913c29249`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:12.221066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d22d58e-c036-475b-8f97-b19aad6a2ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:12.265991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:12.279129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 755/1504] 10.222.0.32 () {58 vars in 1060 bytes} [Wed May 25 14:36:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:36:12.287476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acea1b-f957-499b-81b2-fae68945f496 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.299432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acea1b-f957-499b-81b2-fae68945f496 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.303976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.304612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.304889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.305163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.314404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.316649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.328105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 750/1505] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:36:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:36:12.369864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d6de5-9b0d-4bda-a782-522c175cfb38 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.370155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d6de5-9b0d-4bda-a782-522c175cfb38 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.370425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d6de5-9b0d-4bda-a782-522c175cfb38 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.371802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 756/1506] 10.222.0.32 () {52 vars in 829 bytes} [Wed May 25 14:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:36:12.375574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.442201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff298f3-2dd1-40a0-ad20-db2341ae088f service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.450789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.451697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.452030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.452317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] PATH_INFO: `/v3/users/53f8633f932b4100bdabacb39e84eb4c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.453599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f8633f932b4100bdabacb39e84eb4c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.454625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:12.454625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:36:12.454625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:36:12.455900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.539379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f8633f932b4100bdabacb39e84eb4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:12.540202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f8633f932b4100bdabacb39e84eb4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:12.540876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f8633f932b4100bdabacb39e84eb4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:12.560740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f8633f932b4100bdabacb39e84eb4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:12.573091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f8633f932b4100bdabacb39e84eb4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:12.577196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:12.593187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 751/1507] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:36:12 2022] DELETE /identity/v3/users/53f8633f932b4100bdabacb39e84eb4c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:12.600108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.600627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.601125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.621207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:36:12.622401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:12.683390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 757/1508] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:36:12.694548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:12.696914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:12.756033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99f8c7-cf3d-4006-bcf7-db4f324a2c68 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:12.760124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d0a40-f8a3-4cd5-84a9-493008f986a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:12.760777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d0a40-f8a3-4cd5-84a9-493008f986a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:12.761049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d0a40-f8a3-4cd5-84a9-493008f986a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:12.761320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d0a40-f8a3-4cd5-84a9-493008f986a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:12.770657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d0a40-f8a3-4cd5-84a9-493008f986a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:12.772141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d0a40-f8a3-4cd5-84a9-493008f986a8 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:12.782005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 752/1509] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:36:13.157982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:36:13.159481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:36:13.171501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4f1eb-8806-48f4-865b-063c14042c60 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:36:13.176245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:36:13.178101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:36:13.178395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:36:13.178686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] PATH_INFO: `/v3/projects/3159825930c64d0cb6f2b3823b2536c2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:36:13.180104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3159825930c64d0cb6f2b3823b2536c2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:36:13.181137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:36:13.181137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:36:13.181137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:36:13.182306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:36:13.243129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3159825930c64d0cb6f2b3823b2536c2', 'request_id': 'req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:13.257606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3159825930c64d0cb6f2b3823b2536c2', 'request_id': 'req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:13.264133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3159825930c64d0cb6f2b3823b2536c2', 'request_id': 'req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:13.280503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] The token cache is being invalidate because project 3159825930c64d0cb6f2b3823b2536c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:36:13.280795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:13.281597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d5b9b21-480c-4d70-920c-38c2f0dd74e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:36:13.283295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 758/1510] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:36:13 2022] DELETE /identity/v3/projects/3159825930c64d0cb6f2b3823b2536c2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:36:22.777426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:36:23.277376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:00.151738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e499d1-fdc2-4dbe-a65e-cc7048e974b3 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:00.152370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e499d1-fdc2-4dbe-a65e-cc7048e974b3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:00.152370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e499d1-fdc2-4dbe-a65e-cc7048e974b3 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:00.153688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:00.159121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0495-2a19-4fb9-988b-f4cea45500dd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:00.159568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0495-2a19-4fb9-988b-f4cea45500dd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:00.159685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa0495-2a19-4fb9-988b-f4cea45500dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:00.180222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-56fa0495-2a19-4fb9-988b-f4cea45500dd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:00.181535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56fa0495-2a19-4fb9-988b-f4cea45500dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:00.244500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:00.414906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e499d1-fdc2-4dbe-a65e-cc7048e974b3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:00.416946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f2e499d1-fdc2-4dbe-a65e-cc7048e974b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:00.493069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e499d1-fdc2-4dbe-a65e-cc7048e974b3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:00.499434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0915c6b-70f0-4fea-bc10-0c19d349d611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:00.500092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915c6b-70f0-4fea-bc10-0c19d349d611 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:00.500372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915c6b-70f0-4fea-bc10-0c19d349d611 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:00.500647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0915c6b-70f0-4fea-bc10-0c19d349d611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:00.509069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0915c6b-70f0-4fea-bc10-0c19d349d611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:00.510734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0915c6b-70f0-4fea-bc10-0c19d349d611 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:00.520013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 754/1513] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:01.396325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7615ad2-92fb-4f3d-9743-8d3bb6f08521 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:01.396922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7615ad2-92fb-4f3d-9743-8d3bb6f08521 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:01.396922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7615ad2-92fb-4f3d-9743-8d3bb6f08521 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:01.398309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:01.403391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:01.403843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:01.403967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:01.412792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b9eb1-fc96-438d-9cb4-ff2e6afbdad3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:01.413077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b9eb1-fc96-438d-9cb4-ff2e6afbdad3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:01.413349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b9eb1-fc96-438d-9cb4-ff2e6afbdad3 None None] PATH_INFO: `/v3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:01.414592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 761/1515] 10.222.0.75 () {54 vars in 925 bytes} [Wed May 25 14:38:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:38:01.421475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cfc5c-1005-4a07-86ff-eed464a0c9d2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:01.421761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cfc5c-1005-4a07-86ff-eed464a0c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:01.422036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cfc5c-1005-4a07-86ff-eed464a0c9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:01.423923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:01.425384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:01.433657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 755/1516] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:01.441114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8e1cfc5c-1005-4a07-86ff-eed464a0c9d2 None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:01.442180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8e1cfc5c-1005-4a07-86ff-eed464a0c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:01.503731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 762/1517] 10.222.0.75 () {58 vars in 1015 bytes} [Wed May 25 14:38:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:01.589543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:01.610785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3251d-f49a-452d-a6df-8a620d23a450 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:01.620737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d264d-d776-4b8c-a03f-08ec91e6558b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:01.621380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d264d-d776-4b8c-a03f-08ec91e6558b service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:01.621653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d264d-d776-4b8c-a03f-08ec91e6558b service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:01.621919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d264d-d776-4b8c-a03f-08ec91e6558b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:01.630769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d264d-d776-4b8c-a03f-08ec91e6558b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:01.632398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d264d-d776-4b8c-a03f-08ec91e6558b service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:01.642195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 756/1518] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:10.602335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785c7c3-1378-44a2-9528-c0efc50a8a2e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:10.603291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785c7c3-1378-44a2-9528-c0efc50a8a2e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:10.603291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785c7c3-1378-44a2-9528-c0efc50a8a2e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:10.605362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 763/1519] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 25 14:38:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:10.613995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd836e1-8f3b-4256-9b2e-73e3719a5681 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:10.614463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd836e1-8f3b-4256-9b2e-73e3719a5681 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:10.614565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd836e1-8f3b-4256-9b2e-73e3719a5681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:10.635883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2fd836e1-8f3b-4256-9b2e-73e3719a5681 None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:10.637075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2fd836e1-8f3b-4256-9b2e-73e3719a5681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:10.646270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 757/1520] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 25 14:38:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:10.653990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785c7c3-1378-44a2-9528-c0efc50a8a2e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:10.669865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785c7c3-1378-44a2-9528-c0efc50a8a2e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:10.676207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4f07b-b419-42b3-9cc3-a74622fec3fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:10.676845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4f07b-b419-42b3-9cc3-a74622fec3fe service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:10.677120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4f07b-b419-42b3-9cc3-a74622fec3fe service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:10.677381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4f07b-b419-42b3-9cc3-a74622fec3fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:10.686890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4f07b-b419-42b3-9cc3-a74622fec3fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:10.688499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4f07b-b419-42b3-9cc3-a74622fec3fe service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:10.698694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 764/1521] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:12.919107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c26147-bb23-456a-afda-7c1b2db36822 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:12.919702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c26147-bb23-456a-afda-7c1b2db36822 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:12.919702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c26147-bb23-456a-afda-7c1b2db36822 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:12.921045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:12.927238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b09e35-7e47-4e02-8b18-feca4e05a244 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:12.927915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b09e35-7e47-4e02-8b18-feca4e05a244 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:12.928110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b09e35-7e47-4e02-8b18-feca4e05a244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:12.949719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-01b09e35-7e47-4e02-8b18-feca4e05a244 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:12.950981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01b09e35-7e47-4e02-8b18-feca4e05a244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:12.960312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:13.112263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c26147-bb23-456a-afda-7c1b2db36822 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:13.136460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c26147-bb23-456a-afda-7c1b2db36822 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:13.142587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92583cb5-be2b-4d1a-b547-3cf37d33e69b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:13.143264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92583cb5-be2b-4d1a-b547-3cf37d33e69b service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:13.143543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92583cb5-be2b-4d1a-b547-3cf37d33e69b service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:13.143812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92583cb5-be2b-4d1a-b547-3cf37d33e69b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:13.153066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92583cb5-be2b-4d1a-b547-3cf37d33e69b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:13.154719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92583cb5-be2b-4d1a-b547-3cf37d33e69b service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:13.164697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 759/1524] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:14.028925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940dce-f395-4eb1-8870-e7a67f0b6b83 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:14.029532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940dce-f395-4eb1-8870-e7a67f0b6b83 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:14.029532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940dce-f395-4eb1-8870-e7a67f0b6b83 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:14.030853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 14:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:14.037285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9018c-7788-43f8-b464-e2478a385265 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:14.038013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9018c-7788-43f8-b464-e2478a385265 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:14.038189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9018c-7788-43f8-b464-e2478a385265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:14.059218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-15f9018c-7788-43f8-b464-e2478a385265 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:14.060427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-15f9018c-7788-43f8-b464-e2478a385265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:14.069534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 14:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:14.218597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940dce-f395-4eb1-8870-e7a67f0b6b83 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:14.234898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940dce-f395-4eb1-8870-e7a67f0b6b83 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:14.240553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad32a5-b31d-459d-8f5e-3590994a52c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:14.241183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad32a5-b31d-459d-8f5e-3590994a52c4 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:14.241458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad32a5-b31d-459d-8f5e-3590994a52c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:14.241718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad32a5-b31d-459d-8f5e-3590994a52c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:14.250043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad32a5-b31d-459d-8f5e-3590994a52c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:14.251674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad32a5-b31d-459d-8f5e-3590994a52c4 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:14.261127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 767/1527] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:18.243294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f1da-e135-489a-9cd6-a266d3b2fdc5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:18.243910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f1da-e135-489a-9cd6-a266d3b2fdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:18.243910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f1da-e135-489a-9cd6-a266d3b2fdc5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:18.245233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:38:18.251710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2f37-8e98-4b50-8229-e54e9c4270f5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:18.252425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2f37-8e98-4b50-8229-e54e9c4270f5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:18.252598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2f37-8e98-4b50-8229-e54e9c4270f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:18.278594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1c1c2f37-8e98-4b50-8229-e54e9c4270f5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:18.280386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c1c2f37-8e98-4b50-8229-e54e9c4270f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:18.294292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:18.333948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7f1da-e135-489a-9cd6-a266d3b2fdc5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:18.336324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-08d7f1da-e135-489a-9cd6-a266d3b2fdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:18.399114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7f1da-e135-489a-9cd6-a266d3b2fdc5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:18.403466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6888510-39dd-48ae-8073-1db8e578e5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:18.404109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6888510-39dd-48ae-8073-1db8e578e5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:18.404387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6888510-39dd-48ae-8073-1db8e578e5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:18.404644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6888510-39dd-48ae-8073-1db8e578e5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:18.414852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6888510-39dd-48ae-8073-1db8e578e5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:18.416261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6888510-39dd-48ae-8073-1db8e578e5a9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:18.425724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 762/1530] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:20.609373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05d528-19bc-4232-8a91-640316509903 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:20.610294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05d528-19bc-4232-8a91-640316509903 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:20.610294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05d528-19bc-4232-8a91-640316509903 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:20.612480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:38:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:20.623296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77279ff3-8e8b-4805-a48f-497cb6832d48 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:20.623734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77279ff3-8e8b-4805-a48f-497cb6832d48 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:20.623850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77279ff3-8e8b-4805-a48f-497cb6832d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:20.647914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-77279ff3-8e8b-4805-a48f-497cb6832d48 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:20.649123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-77279ff3-8e8b-4805-a48f-497cb6832d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:20.658678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:20.711082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05d528-19bc-4232-8a91-640316509903 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:20.729487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05d528-19bc-4232-8a91-640316509903 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:20.734000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05567b5-5579-415f-880b-9334ef3482c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:20.734635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05567b5-5579-415f-880b-9334ef3482c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:20.734951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05567b5-5579-415f-880b-9334ef3482c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:20.735224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05567b5-5579-415f-880b-9334ef3482c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:20.744507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05567b5-5579-415f-880b-9334ef3482c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:20.745923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05567b5-5579-415f-880b-9334ef3482c5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:20.755333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 770/1533] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:24.508594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf309-4401-479b-b9b4-48f813c4d651 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:24.509504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf309-4401-479b-b9b4-48f813c4d651 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:24.509504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cf309-4401-479b-b9b4-48f813c4d651 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:24.511807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:38:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:24.517314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f32d8-60fc-4f89-b02e-b67c0c918afc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:24.517733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f32d8-60fc-4f89-b02e-b67c0c918afc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:24.517839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f32d8-60fc-4f89-b02e-b67c0c918afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:24.525112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:24.539361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-743f32d8-60fc-4f89-b02e-b67c0c918afc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:24.540550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-743f32d8-60fc-4f89-b02e-b67c0c918afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:24.549230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:24.815451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf309-4401-479b-b9b4-48f813c4d651 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:24.817537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b64cf309-4401-479b-b9b4-48f813c4d651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:24.824309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:24.880200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cf309-4401-479b-b9b4-48f813c4d651 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:24.884519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e728b19-ff95-4d0d-b30d-a1868a22bcb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:24.885169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728b19-ff95-4d0d-b30d-a1868a22bcb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:24.885456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728b19-ff95-4d0d-b30d-a1868a22bcb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:24.885729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728b19-ff95-4d0d-b30d-a1868a22bcb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:24.895728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728b19-ff95-4d0d-b30d-a1868a22bcb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:24.897180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728b19-ff95-4d0d-b30d-a1868a22bcb5 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:24.907076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 765/1536] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:27.107007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8243ad-2a5a-45ed-be28-c2333bce8a23 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:27.107590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8243ad-2a5a-45ed-be28-c2333bce8a23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:27.107590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8243ad-2a5a-45ed-be28-c2333bce8a23 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:27.108956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 14:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:38:27.115447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314269-78f1-400e-802f-59e0eafe860b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:27.116151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314269-78f1-400e-802f-59e0eafe860b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:27.116332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69314269-78f1-400e-802f-59e0eafe860b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:27.137080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-69314269-78f1-400e-802f-59e0eafe860b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:38:27.138313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69314269-78f1-400e-802f-59e0eafe860b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:27.146630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 14:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:38:27.413202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8243ad-2a5a-45ed-be28-c2333bce8a23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:38:27.415428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1e8243ad-2a5a-45ed-be28-c2333bce8a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:38:27.429540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8243ad-2a5a-45ed-be28-c2333bce8a23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:38:27.433644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9b25f-11ed-414a-ae2b-1e9aad4e97c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:38:27.434281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b25f-11ed-414a-ae2b-1e9aad4e97c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:38:27.434557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b25f-11ed-414a-ae2b-1e9aad4e97c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:38:27.434874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9b25f-11ed-414a-ae2b-1e9aad4e97c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:38:27.444143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9b25f-11ed-414a-ae2b-1e9aad4e97c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:38:27.445653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9b25f-11ed-414a-ae2b-1e9aad4e97c9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:38:27.455001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 773/1539] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:38:37.132073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:38:37.451365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:01.456885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbc016-259c-4c9d-a81c-7904d81cf875 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.458179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbc016-259c-4c9d-a81c-7904d81cf875 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.458784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbc016-259c-4c9d-a81c-7904d81cf875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.505586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d1fbc016-259c-4c9d-a81c-7904d81cf875 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.508127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d1fbc016-259c-4c9d-a81c-7904d81cf875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.525231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 767/1540] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.537557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.538623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.539144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.565001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.566634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.574645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c75e4-74e1-45cb-a332-4f40cecfa420 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.575383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c75e4-74e1-45cb-a332-4f40cecfa420 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.575883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c75e4-74e1-45cb-a332-4f40cecfa420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.592725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 774/1541] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.606950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e70c75e4-74e1-45cb-a332-4f40cecfa420 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.608898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e70c75e4-74e1-45cb-a332-4f40cecfa420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.619580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 768/1542] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.623064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.635088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d622eb4-e643-4a57-bf2b-4fccd4aac940 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.641683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.644996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.645870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.646641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.651280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.651878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.652373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.653511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:01.654038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:01.656775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9bf92-dcb1-43d5-8698-15689515457f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:01.694289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.696271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 775/1543] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:01.696562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.697466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.697754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.698034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.711568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 769/1544] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 25 14:40:01.713115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.729243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa8668-c52f-423c-b45d-7181e77abe7a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.734687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf092306-aa8e-431c-80c9-df98dfb1ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.735541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf092306-aa8e-431c-80c9-df98dfb1ae3a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.735751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.735887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf092306-aa8e-431c-80c9-df98dfb1ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.736085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf092306-aa8e-431c-80c9-df98dfb1ae3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.736790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf092306-aa8e-431c-80c9-df98dfb1ae3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:01.736977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.738739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf092306-aa8e-431c-80c9-df98dfb1ae3a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:01.746151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 776/1545] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:01.747106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.757777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ce0de-1dcc-4a67-9546-0e8079d623ff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.762874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.763917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.764245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.764423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.765444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:01.765879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:01.767704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae6145-e655-4e55-8407-c1306a335ecb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:01.775673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 770/1546] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/projects => generated 391 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:01.777910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71bd9e-d5f7-4310-8e9d-30bd406badd5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.778682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71bd9e-d5f7-4310-8e9d-30bd406badd5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.783827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 777/1547] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:01.784003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71bd9e-d5f7-4310-8e9d-30bd406badd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.784100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d0a8-0619-4036-bf73-9b191e2b86eb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.784100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d0a8-0619-4036-bf73-9b191e2b86eb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.784100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d0a8-0619-4036-bf73-9b191e2b86eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.803648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aec4d0a8-0619-4036-bf73-9b191e2b86eb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.804840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0f71bd9e-d5f7-4310-8e9d-30bd406badd5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.804988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aec4d0a8-0619-4036-bf73-9b191e2b86eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.806201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0f71bd9e-d5f7-4310-8e9d-30bd406badd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.872224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 778/1548] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.872709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 771/1549] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.874339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588056dc-2fce-44c0-b4c6-998bbdc328bd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.874638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588056dc-2fce-44c0-b4c6-998bbdc328bd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.874940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588056dc-2fce-44c0-b4c6-998bbdc328bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.882450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.882741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.883038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.895804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-588056dc-2fce-44c0-b4c6-998bbdc328bd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.897013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-588056dc-2fce-44c0-b4c6-998bbdc328bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.903275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.904774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.906216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 772/1550] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.907147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.907496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.907688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.913240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 779/1551] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.914128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.924639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7671f-e178-4671-b2ac-c494ff0d16b6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.927654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.928873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.929221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5d0d5-1283-4cd5-b01c-bf481808df97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.929884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5d0d5-1283-4cd5-b01c-bf481808df97 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.930159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5d0d5-1283-4cd5-b01c-bf481808df97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.930421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5d0d5-1283-4cd5-b01c-bf481808df97 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.931179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5d0d5-1283-4cd5-b01c-bf481808df97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:01.933074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5d0d5-1283-4cd5-b01c-bf481808df97 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:01.936814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 773/1552] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.937578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:01.947535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebecc3-ff19-4b6d-8770-8083065f5ee0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:01.952678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccadc2-985f-4113-8901-ad4360df6038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:01.953422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccadc2-985f-4113-8901-ad4360df6038 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.953679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccadc2-985f-4113-8901-ad4360df6038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.953949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccadc2-985f-4113-8901-ad4360df6038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.954662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccadc2-985f-4113-8901-ad4360df6038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:01.956467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 780/1553] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:01.957760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ef306-73ed-4ae8-bd60-5adbf0631025 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.958044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ef306-73ed-4ae8-bd60-5adbf0631025 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.958154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccadc2-985f-4113-8901-ad4360df6038 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:01.958309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ef306-73ed-4ae8-bd60-5adbf0631025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.978788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-682ef306-73ed-4ae8-bd60-5adbf0631025 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:01.980425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-682ef306-73ed-4ae8-bd60-5adbf0631025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:01.988810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 774/1554] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:01.994962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 781/1555] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:01.995119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.995119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.995285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e22400-3430-4774-9f0f-69e02e7c9c09 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:01.995374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:01.995472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e22400-3430-4774-9f0f-69e02e7c9c09 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:01.995472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e22400-3430-4774-9f0f-69e02e7c9c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.011526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:02.012738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:02.013863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-72e22400-3430-4774-9f0f-69e02e7c9c09 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:02.015272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72e22400-3430-4774-9f0f-69e02e7c9c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:02.071467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 775/1556] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:02.072402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.073254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 782/1557] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:02.074818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d949dc2-a8c8-489e-933d-2dd3645b5b91 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.075448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d949dc2-a8c8-489e-933d-2dd3645b5b91 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.075727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d949dc2-a8c8-489e-933d-2dd3645b5b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.085723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c572ff-60af-4af9-a772-25586ff614fc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.090284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.090968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.091250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.091519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.092248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.092582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.093297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2d949dc2-a8c8-489e-933d-2dd3645b5b91 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:02.093687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 14:40:02.093687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:40:02.093687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:40:02.094473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d949dc2-a8c8-489e-933d-2dd3645b5b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:02.094584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.102196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 783/1558] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:02.103433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.103708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.103969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.109289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 776/1559] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.109289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.120217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0903f5-296d-480a-93b6-40098006e23e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.125117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c590f-79cc-4647-bb42-9813dac604bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.125784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c590f-79cc-4647-bb42-9813dac604bd admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.126065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c590f-79cc-4647-bb42-9813dac604bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.126306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c590f-79cc-4647-bb42-9813dac604bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.127075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c590f-79cc-4647-bb42-9813dac604bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.127190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:02.128321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:02.128714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c590f-79cc-4647-bb42-9813dac604bd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.134165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.136464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 784/1560] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:02.137464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.141113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 777/1561] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:02.142730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.144023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.144023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.149178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eff31f-a420-4cc1-8ea3-a6b161e75a84 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.155120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.155795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.156072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.156342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.157029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.158971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.162185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:02.163442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:02.172414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 778/1562] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:02.174421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.177173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.183152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 785/1563] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.183152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.186859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa1fb-de24-4bf6-a6c5-5bffa05d2baf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.190910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c26d8f-06a3-488d-922e-ebc3c4aecacc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.191885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.192546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.192901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.193173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.193898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.194201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.195832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.195966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.196488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.196773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.197049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.197778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.198492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.200494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.209044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 779/1564] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.209897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.215895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 786/1565] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.216831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.226578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81884-486d-420f-86a0-7822bac1ff6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.227650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293f2c5-3ecd-4073-9eb2-a3ed50a4a78d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.231568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.232237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.232573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.232717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.232933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.233242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.233599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.233868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.234039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.234591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.234932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.235099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:40:02.235099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:40:02.235099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:40:02.236404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.236555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.248766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 780/1566] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.249694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.265143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 787/1567] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.266083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.269936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34614c09-b9ef-4a6e-ac1c-0bbf23458a7d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.328208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.328890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.329167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.329479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438168a-760f-421b-9300-23aef3825de6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.329617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.330153 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.330460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.332676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.334308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.335011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.335286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.335565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.336283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.337900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.352831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 781/1568] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.352831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.357065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 788/1569] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:02.358326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.358522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d074ec32-ae3b-44ae-afd2-8a1b0c649233 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.366522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.367175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.367458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.367728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.368457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:02.368761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.370430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.371953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd9994-42c2-4a69-a0df-9cf063940b31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.376634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.377325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.377602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.377872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.378597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.380196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.383578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 782/1570] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:02.384497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.394269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3409-9ba6-4286-b84a-ecb6463888d5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.394966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 789/1571] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:02.395666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.401581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.402435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.402804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.403163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.403904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.406389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.406764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb1c76-e51a-4736-b2f3-fe269c75783b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.411581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.412239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.412521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.412794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.413485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.415413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.435000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 783/1572] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.436011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.447481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 790/1573] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.448415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.504886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba74338-2006-4cd6-bb3b-a340dd054c96 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.514326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af40c8e-e8d0-49c5-8156-14137ac70196 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.518957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.519627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.519903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.520173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.520861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.522793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.534141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.546278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 791/1574] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.546278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.554093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e76d1-deec-49fa-9773-356eb2b7ec6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.558647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.559328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.559622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.559878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.560570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.562553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.564668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.565348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.565632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.565909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.566598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.568558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.577343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.599739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 792/1575] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.599739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.604899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 784/1576] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.605738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.669439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1889267-f858-442e-aa22-6ec1d013ca2e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.670375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ee57a-850b-4e81-9d25-5f5257f83b81 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.674688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.675241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.675688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.676039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.676379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.676678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.677035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.677253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.677437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] PATH_INFO: `/v3/projects/72b4ce05423043b69e280264439ddbe6/users/73d50022c5f94444afc67ea810e4d6fe/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.679383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.695349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b4ce05423043b69e280264439ddbe6, user_id=73d50022c5f94444afc67ea810e4d6fe, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.696408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:40:02.696408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:40:02.696408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:40:02.698553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.710646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 785/1577] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.711622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.715108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 793/1578] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:02 2022] PUT /identity/v3/projects/72b4ce05423043b69e280264439ddbe6/users/73d50022c5f94444afc67ea810e4d6fe/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:02.716487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.773333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923c602-cba0-404b-bd97-d907093842dd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.777788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.778465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.778740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.779363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.780437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.781652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be71c5e-d9f2-44bd-8e46-5833e15e21f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.781914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.788798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.789501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.789773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.790041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.790736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.792703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.793141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 786/1579] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:02.797624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.807614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca2e05-5e46-4519-b471-feb33d5b7f5d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.812905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.813575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.813878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.814149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.814871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.815895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:40:02.815895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:40:02.815895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:40:02.817099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.817790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 794/1580] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.818658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.828335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f86a1-3dc2-4bb8-8f49-2e818bfefb6b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.833030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.833715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.833962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.834265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.834977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.837215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.843192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 787/1581] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.844147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.850060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 795/1582] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:02.850914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.853940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a060a-9e46-434d-bbb9-ede3f0501c5a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.859043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.859731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.859979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.860286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.861094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.861633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762e636-7d65-4ab5-9a14-422f8cfca39b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.863166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.866639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a8a89-66a8-4e67-adef-376bb89e6126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.867417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a8a89-66a8-4e67-adef-376bb89e6126 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.867792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a8a89-66a8-4e67-adef-376bb89e6126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.868157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a8a89-66a8-4e67-adef-376bb89e6126 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.868952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a8a89-66a8-4e67-adef-376bb89e6126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.871026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a8a89-66a8-4e67-adef-376bb89e6126 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.889274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 788/1583] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.889274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.893285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:02.897154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b58a58-d6f1-4852-afd7-60ec8d87e9fd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.900996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 796/1584] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.900996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.900996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.900996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.905452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.906809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.907101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.907384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.908076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.910030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.918050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:02.919296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:02.928693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 797/1585] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:02.929699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.936831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 789/1586] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:02 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:02.938075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.939729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21feec6b-f576-4d9c-bcf2-f800f16a5556 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.944899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.945851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.946128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.946394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] PATH_INFO: `/v3/projects/ff4a889141fb4ed7bf679ca8c4623d16/users/18ff373917f2471d9e811c747a718f6a/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.952667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233f323-0fc3-487c-8110-5198ec4cc91d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.958740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:02.959455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:02.959732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:02.960005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:02.960729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.962020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4a889141fb4ed7bf679ca8c4623d16, user_id=18ff373917f2471d9e811c747a718f6a, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:02.962386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.964897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:02.973731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 790/1587] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:02.977491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.980618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 798/1588] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:02 2022] PUT /identity/v3/projects/ff4a889141fb4ed7bf679ca8c4623d16/users/18ff373917f2471d9e811c747a718f6a/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:02.984403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:02.990386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d0ad3-5d0e-4a08-aa06-b681ae215e45 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:02.997532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:03.049478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.050068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.050348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.050622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.051391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.052995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2b803-d7c1-46a4-bbc5-f1fc09df7faf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.053145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.057537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.058205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.058486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.058773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.059551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.061299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.065919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 791/1589] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.065919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.071487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 799/1590] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.072153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.078560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1420a-a0e9-46df-9204-ab0a09f34e3a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.081786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e59cc8-f46c-4ed2-a1e4-9fb04f23479c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.083579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.084257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.084531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.084803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.085518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.086469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.087146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.087273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.087426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.087698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.088421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.089982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.098579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 792/1591] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.099425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.100568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 800/1592] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.101384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.110171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b3fa3-6a6a-44a8-83bb-2676626e6e6e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.115831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.116483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.116759 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.117027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.117738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.119365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.129636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 793/1593] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.130580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.143207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6ecda-cd89-4efd-852c-bbbd025c5e00 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.148370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9be6ee-0bdc-4fd6-859a-10c7a80abca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.149048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9be6ee-0bdc-4fd6-859a-10c7a80abca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.149307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9be6ee-0bdc-4fd6-859a-10c7a80abca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.149571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9be6ee-0bdc-4fd6-859a-10c7a80abca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.150289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9be6ee-0bdc-4fd6-859a-10c7a80abca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.151916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9be6ee-0bdc-4fd6-859a-10c7a80abca5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.156713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc3f8d-c7a1-49b9-9393-7960dc10e74d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.160322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.160965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.161272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.161572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.166381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 794/1594] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.166381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.166381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.166381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.174960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.176390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.185994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 801/1595] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:03.186927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.192931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] MFA Rules not processed for user `18ff373917f2471d9e811c747a718f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:03.194082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:03.196485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14279ca-dad2-4bc8-84ef-7c0e4180b0d1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.201445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.202134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.202468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.202739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.203694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.205334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.222330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 802/1596] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.223280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.234459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d046f5-672a-4b66-845c-7ffa083df0c7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.239951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.240870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.241146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.241414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] PATH_INFO: `/v3/projects/8e6e9d7564324153a7edc2b5f04c4c6c/users/c00d1d4562584ba69a7a9a5fb016b2ba/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.261310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6e9d7564324153a7edc2b5f04c4c6c, user_id=c00d1d4562584ba69a7a9a5fb016b2ba, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.264621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.274286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 795/1597] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:03.274286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.284783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c593d11-8396-4216-b61f-2c4f8fa433fc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.290851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.291654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.292022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.292396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.293356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.295326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.301419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 803/1598] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:03 2022] PUT /identity/v3/projects/8e6e9d7564324153a7edc2b5f04c4c6c/users/c00d1d4562584ba69a7a9a5fb016b2ba/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:03.302989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.312602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 796/1599] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.313688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.326234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1acb-9fef-40b8-a6bf-066e49bf2fd0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.341500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:03.388636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c78f61-a4f5-4072-a47f-6b5bb89e22ff admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.395690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.396424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.396701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.396971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.397999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.399811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.400309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.400998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.401331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.401546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.402282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.402909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:03.403945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.418225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 797/1600] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.418470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 804/1601] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:03.419311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.419423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.430238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a660231-4777-4ac7-a7ae-f4cafcae29a3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.430434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02dd91-32d5-43a8-8dce-9c9013a41f6f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.435138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5191e4f-d8a8-403a-a0d4-9d02ccda47af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.435491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b4b6f-4931-48c3-848f-cdc0a8cda20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.436142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191e4f-d8a8-403a-a0d4-9d02ccda47af admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.436391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b4b6f-4931-48c3-848f-cdc0a8cda20b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.436490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191e4f-d8a8-403a-a0d4-9d02ccda47af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.436782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b4b6f-4931-48c3-848f-cdc0a8cda20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.436888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5191e4f-d8a8-403a-a0d4-9d02ccda47af admin admin] PATH_INFO: `/v3/projects/a4b0cd8531734cd6a2a9c5daba97e7e9/users/c73c7358f0114c0fbdb77d29ce3a68d3/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.437209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b4b6f-4931-48c3-848f-cdc0a8cda20b admin admin] PATH_INFO: `/v3/projects/16613c1e95ff4565a9ed9393295064a2/users/2401371a775743c29c16406cb9df5d28/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.454865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b4b6f-4931-48c3-848f-cdc0a8cda20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16613c1e95ff4565a9ed9393295064a2, user_id=2401371a775743c29c16406cb9df5d28, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.455286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191e4f-d8a8-403a-a0d4-9d02ccda47af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b0cd8531734cd6a2a9c5daba97e7e9, user_id=c73c7358f0114c0fbdb77d29ce3a68d3, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.457616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b4b6f-4931-48c3-848f-cdc0a8cda20b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.458153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5191e4f-d8a8-403a-a0d4-9d02ccda47af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.475450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 798/1602] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:03 2022] PUT /identity/v3/projects/16613c1e95ff4565a9ed9393295064a2/users/2401371a775743c29c16406cb9df5d28/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:03.477440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.477850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.488810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 805/1603] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:03 2022] PUT /identity/v3/projects/a4b0cd8531734cd6a2a9c5daba97e7e9/users/c73c7358f0114c0fbdb77d29ce3a68d3/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:03.489035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.489183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c0b2-cc39-4c8c-b930-a69e170dfd99 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.489183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c0b2-cc39-4c8c-b930-a69e170dfd99 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.489183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97c0b2-cc39-4c8c-b930-a69e170dfd99 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.489183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 806/1604] 10.222.0.32 () {54 vars in 961 bytes} [Wed May 25 14:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:03.489183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c0b2-cc39-4c8c-b930-a69e170dfd99 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.505631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:03.506998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:03.570291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97c0b2-cc39-4c8c-b930-a69e170dfd99 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.575153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.576264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.576719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.577095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] PATH_INFO: `/v3/projects/4753380d566d4307bf8d1bc0eeca8c15/users/8d8f33f0c29146c4bbbcc50c65336caa/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.591825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 799/1605] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:03.591825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.600462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4753380d566d4307bf8d1bc0eeca8c15, user_id=8d8f33f0c29146c4bbbcc50c65336caa, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.604155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.605415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af82a-0abf-4520-bd74-f564343f3f90 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.610202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321d703-039c-4b0d-a2e7-a07edd82c077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.611276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321d703-039c-4b0d-a2e7-a07edd82c077 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.611633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321d703-039c-4b0d-a2e7-a07edd82c077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.612052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321d703-039c-4b0d-a2e7-a07edd82c077 admin admin] PATH_INFO: `/v3/projects/2f93c08bb4234073b8dd996119d06701/users/175d6fedca25472796216b15ecd3bc1a/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.622072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 807/1606] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:03 2022] PUT /identity/v3/projects/4753380d566d4307bf8d1bc0eeca8c15/users/8d8f33f0c29146c4bbbcc50c65336caa/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:03.623060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.629840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321d703-039c-4b0d-a2e7-a07edd82c077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f93c08bb4234073b8dd996119d06701, user_id=175d6fedca25472796216b15ecd3bc1a, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.634037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321d703-039c-4b0d-a2e7-a07edd82c077 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:03.651060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 800/1607] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:03 2022] PUT /identity/v3/projects/2f93c08bb4234073b8dd996119d06701/users/175d6fedca25472796216b15ecd3bc1a/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:03.652693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1c98-a7fe-4604-a45a-c455e5f53602 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.652980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1c98-a7fe-4604-a45a-c455e5f53602 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.653255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb1c98-a7fe-4604-a45a-c455e5f53602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.674092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-08cb1c98-a7fe-4604-a45a-c455e5f53602 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:03.675432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-08cb1c98-a7fe-4604-a45a-c455e5f53602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:03.708996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239f7b-6913-4d9a-a20d-fc225199a8fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:03.740859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 801/1608] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:03.742174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851520db-1d04-4690-b68a-8f4f855d6bbd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.742534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851520db-1d04-4690-b68a-8f4f855d6bbd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.742939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851520db-1d04-4690-b68a-8f4f855d6bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.762594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:03.770932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-851520db-1d04-4690-b68a-8f4f855d6bbd None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:03.773251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-851520db-1d04-4690-b68a-8f4f855d6bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:03.786710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea33b89-ebdf-4ba0-b2d8-d91c78ddded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:03.788733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33b89-ebdf-4ba0-b2d8-d91c78ddded0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.788733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33b89-ebdf-4ba0-b2d8-d91c78ddded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.789059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33b89-ebdf-4ba0-b2d8-d91c78ddded0 admin admin] PATH_INFO: `/v3/projects/7f112e14f33a4cd0af6e07461d8fdb4a/users/2d2256afe4094555b206050fb1e09365/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.800765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:03.852628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 802/1609] 10.222.0.32 () {58 vars in 1058 bytes} [Wed May 25 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:03.854063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:03.854376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:03.854657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:03.885458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] MFA Rules not processed for user `c73c7358f0114c0fbdb77d29ce3a68d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:03.887331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:03.969411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 803/1610] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:03.969411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:03.969411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:03.978636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea33b89-ebdf-4ba0-b2d8-d91c78ddded0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f112e14f33a4cd0af6e07461d8fdb4a, user_id=2d2256afe4094555b206050fb1e09365, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:03.981478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea33b89-ebdf-4ba0-b2d8-d91c78ddded0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.006894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 808/1611] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:03 2022] PUT /identity/v3/projects/7f112e14f33a4cd0af6e07461d8fdb4a/users/2d2256afe4094555b206050fb1e09365/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:04.008258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4d7af-92ed-4b98-9520-ae1e8fa73dc3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.008543 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4d7af-92ed-4b98-9520-ae1e8fa73dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.008815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4d7af-92ed-4b98-9520-ae1e8fa73dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.038037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3eb4d7af-92ed-4b98-9520-ae1e8fa73dc3 None None] MFA Rules not processed for user `8d8f33f0c29146c4bbbcc50c65336caa`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:04.039303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3eb4d7af-92ed-4b98-9520-ae1e8fa73dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:04.057885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8294cf-ce10-45de-b7dd-3fa12b5491cf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.109079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:04.115936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 809/1612] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:04.115936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.115936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.115936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.121444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.121932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.122288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.122695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.133331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:04.134257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:04.209817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.211784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.217052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 810/1613] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:04.219263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.230125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4d89e-bc99-409f-9809-640038381edc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.230563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 804/1614] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:04.231417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.234986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.235859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.236325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.236732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.252631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.254316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.268906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 811/1615] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:04.271402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.306690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4ab8a-8221-4420-8d19-9ebb11992fe7 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.319772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1a8ed-0acb-44c5-aa9e-80b197762d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.321997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1a8ed-0acb-44c5-aa9e-80b197762d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.322285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1a8ed-0acb-44c5-aa9e-80b197762d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.324208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1a8ed-0acb-44c5-aa9e-80b197762d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.326348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1a8ed-0acb-44c5-aa9e-80b197762d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.328464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1a8ed-0acb-44c5-aa9e-80b197762d9d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.340078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf3a5-8ce2-4611-b451-3fd32d7b4088 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.347753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c250d-d96e-4d22-89df-d2222d15e967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.348564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c250d-d96e-4d22-89df-d2222d15e967 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.348956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c250d-d96e-4d22-89df-d2222d15e967 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.349295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c250d-d96e-4d22-89df-d2222d15e967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.387247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 812/1616] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:04.388480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.388834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.389163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.430029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:04.431615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:04.446771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c250d-d96e-4d22-89df-d2222d15e967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.448847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c250d-d96e-4d22-89df-d2222d15e967 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.462742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 805/1617] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:04.465568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5792484-c6d9-4c03-94cc-c61fdd235ec4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.466815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5792484-c6d9-4c03-94cc-c61fdd235ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.467194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5792484-c6d9-4c03-94cc-c61fdd235ec4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.468902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 806/1618] 10.222.0.32 () {54 vars in 961 bytes} [Wed May 25 14:40:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:04.470039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5792484-c6d9-4c03-94cc-c61fdd235ec4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.513335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:04.550106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:04.562032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 813/1619] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:04.562923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.585576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5406e-5f67-4f7a-a19b-a0f68f79e316 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.596255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc9dc4-39d1-425b-9b78-ccac522923b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.596999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9dc4-39d1-425b-9b78-ccac522923b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.597421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9dc4-39d1-425b-9b78-ccac522923b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.597715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9dc4-39d1-425b-9b78-ccac522923b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.598429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc9dc4-39d1-425b-9b78-ccac522923b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.600425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc9dc4-39d1-425b-9b78-ccac522923b5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.602093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5792484-c6d9-4c03-94cc-c61fdd235ec4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.610494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.611248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.611574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.611900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.628021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 814/1620] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/users => generated 297 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:04.628021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.628021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.628021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.635523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.637215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.650206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:04.651474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:04.664974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 807/1621] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:04.666107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.683244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c7502-04a2-4e0b-a148-b3b9c1e1b33e service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.702049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.703107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.703545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.703975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.730275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.732099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.754361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 808/1622] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:04.755578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.768409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a7d79-5008-42b3-b0e4-90ab3cabd830 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.774866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a3a41-c891-49e5-ab48-60b70ace0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.775641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a3a41-c891-49e5-ab48-60b70ace0832 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.775931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a3a41-c891-49e5-ab48-60b70ace0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.776280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a3a41-c891-49e5-ab48-60b70ace0832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.777025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a3a41-c891-49e5-ab48-60b70ace0832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.778857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a3a41-c891-49e5-ab48-60b70ace0832 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.794074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 809/1623] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:04.794074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff518b06-eccf-47bf-9e31-62855e955d87 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.794621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff518b06-eccf-47bf-9e31-62855e955d87 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.794721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff518b06-eccf-47bf-9e31-62855e955d87 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.801267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 810/1624] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 25 14:40:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:04.802380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff518b06-eccf-47bf-9e31-62855e955d87 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.818539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff518b06-eccf-47bf-9e31-62855e955d87 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:04.824489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e620f-e36e-4970-916e-543ca18244c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:04.825550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e620f-e36e-4970-916e-543ca18244c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.825837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e620f-e36e-4970-916e-543ca18244c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.826111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e620f-e36e-4970-916e-543ca18244c3 admin admin] PATH_INFO: `/v3/projects/6620f46062ee4a658dd39d840fdfd919/users/3295f3a2e9d84affbdba5c0b3209ff0f/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.846660 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e620f-e36e-4970-916e-543ca18244c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6620f46062ee4a658dd39d840fdfd919, user_id=3295f3a2e9d84affbdba5c0b3209ff0f, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:04.849685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e620f-e36e-4970-916e-543ca18244c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:04.875055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 811/1625] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:04 2022] PUT /identity/v3/projects/6620f46062ee4a658dd39d840fdfd919/users/3295f3a2e9d84affbdba5c0b3209ff0f/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:04.876581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:04.876870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:04.877141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:04.915667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 815/1626] 10.222.0.32 () {58 vars in 1058 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:04.916710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:04.918201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:04.920365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:04.936061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588132a-e9cb-4f38-a8e2-a9872f88d913 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.066807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.067520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.067833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.068149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.068887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.070603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.090271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 816/1627] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:05.091406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.098202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 812/1628] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:40:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:05.099325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.116685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca663f-8ba3-41ea-b93f-5336c684baed admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.124063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c64f22-8aa9-4dd3-9756-6e8d531b16a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.124406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b3345-d3ea-4930-9348-c4c4d2b68357 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.125223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64f22-8aa9-4dd3-9756-6e8d531b16a5 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.125730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64f22-8aa9-4dd3-9756-6e8d531b16a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.126333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c64f22-8aa9-4dd3-9756-6e8d531b16a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.131631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.132561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.132848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.133560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] PATH_INFO: `/v3/projects/6620f46062ee4a658dd39d840fdfd919/users/3295f3a2e9d84affbdba5c0b3209ff0f/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.137370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6620f46062ee4a658dd39d840fdfd919, user_id=3295f3a2e9d84affbdba5c0b3209ff0f, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.142177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.159778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 813/1629] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:05 2022] PUT /identity/v3/projects/6620f46062ee4a658dd39d840fdfd919/users/3295f3a2e9d84affbdba5c0b3209ff0f/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:05.160640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.212180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:05.225643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64f22-8aa9-4dd3-9756-6e8d531b16a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.228642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c64f22-8aa9-4dd3-9756-6e8d531b16a5 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.247403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cab4d5-b42e-4b06-b6ec-4e7beef676d5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.248969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 817/1630] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:05.257168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241652-afdb-4568-851a-0b8656f02e08 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.257532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241652-afdb-4568-851a-0b8656f02e08 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.257929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241652-afdb-4568-851a-0b8656f02e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.281857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.283983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.284329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.284521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.355071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7e241652-afdb-4568-851a-0b8656f02e08 None None] MFA Rules not processed for user `3295f3a2e9d84affbdba5c0b3209ff0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:05.358956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e241652-afdb-4568-851a-0b8656f02e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:05.371555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:05.474214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.476915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.499741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 814/1631] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:05.562812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 818/1632] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:05.581548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.593895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a5cc0-ef28-4e6c-8559-8773cea0ce59 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.614063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53722fec-4888-4e77-a0b9-bd68b17cf111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.615332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722fec-4888-4e77-a0b9-bd68b17cf111 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.617768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722fec-4888-4e77-a0b9-bd68b17cf111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.619292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53722fec-4888-4e77-a0b9-bd68b17cf111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.648306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53722fec-4888-4e77-a0b9-bd68b17cf111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.651894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53722fec-4888-4e77-a0b9-bd68b17cf111 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.684693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 815/1633] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:05.778255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25c960-e75b-41c4-8857-3c846ac15441 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.779719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25c960-e75b-41c4-8857-3c846ac15441 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.781320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25c960-e75b-41c4-8857-3c846ac15441 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.783674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 819/1634] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 25 14:40:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:05.791273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.796543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.796543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.825828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:05.827419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:05.853530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 816/1635] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:05.886538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25c960-e75b-41c4-8857-3c846ac15441 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:05.923568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25c960-e75b-41c4-8857-3c846ac15441 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:05.937320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:05.939544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:05.941248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:05.942469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:05.965860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:05.969028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:05.994584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 820/1636] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:07.271328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.290067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb09fe-cdc8-4289-b3b5-14cc779036d8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.306870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3295f3a2e9d84affbdba5c0b3209ff0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6620f46062ee4a658dd39d840fdfd919', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.310530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.313765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.314293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.315398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:07.315965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.318828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.355162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 817/1637] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:07.360150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.377853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2d6e-8f5c-4f3d-a5de-418c92a0d413 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.389895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3295f3a2e9d84affbdba5c0b3209ff0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6620f46062ee4a658dd39d840fdfd919', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.393420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.393715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.394000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.398439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.405926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.453920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 821/1638] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:07 2022] POST /identity/v3/projects => generated 355 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:07.458338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.476167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d94381-17e8-477c-a542-ad040079f978 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.596861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3295f3a2e9d84affbdba5c0b3209ff0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6620f46062ee4a658dd39d840fdfd919', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.597633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.598662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.599292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.600178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:07.600544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.602448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.621949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 818/1639] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:07.668736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:07.681267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c271b2a-5e81-4a4d-b2d8-47da36f2b3eb tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:07.689382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3295f3a2e9d84affbdba5c0b3209ff0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6620f46062ee4a658dd39d840fdfd919', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:07.691022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:07.691505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:07.691945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] PATH_INFO: `/v3/projects/e30369f0c12f4506a7edcd1f6e87d974` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:07.693500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e30369f0c12f4506a7edcd1f6e87d974)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:07.697447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:07.814530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30369f0c12f4506a7edcd1f6e87d974', 'request_id': 'req-9c8c2099-60ac-4d97-af43-a97342866b56'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:07.828768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30369f0c12f4506a7edcd1f6e87d974', 'request_id': 'req-9c8c2099-60ac-4d97-af43-a97342866b56'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:07.839620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30369f0c12f4506a7edcd1f6e87d974', 'request_id': 'req-9c8c2099-60ac-4d97-af43-a97342866b56'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:07.870387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] The token cache is being invalidate because project e30369f0c12f4506a7edcd1f6e87d974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:07.870955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:07.872096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c8c2099-60ac-4d97-af43-a97342866b56 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:07.876376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 822/1640] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:07 2022] DELETE /identity/v3/projects/e30369f0c12f4506a7edcd1f6e87d974 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:08.360255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:08.362175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:08.516544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d16dd7-0741-4680-95e6-990fe1a53f48 tempest-VolumeQuotaClassesTest-1839208539 tempest-VolumeQuotaClassesTest-1839208539-project-admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:08.544327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:08.651907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:08.654731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.655143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.655380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] PATH_INFO: `/v3/users/c73c7358f0114c0fbdb77d29ce3a68d3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.656859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73c7358f0114c0fbdb77d29ce3a68d3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:08.659860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:08.752782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf72c-a866-4284-abf9-58a157a80570 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.754314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf72c-a866-4284-abf9-58a157a80570 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.755107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf72c-a866-4284-abf9-58a157a80570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.760456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73c7358f0114c0fbdb77d29ce3a68d3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:08.762193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73c7358f0114c0fbdb77d29ce3a68d3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:08.763043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73c7358f0114c0fbdb77d29ce3a68d3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:08.794921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73c7358f0114c0fbdb77d29ce3a68d3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:08.807303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9093a64f-37a6-469f-a420-e58b60a1726a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73c7358f0114c0fbdb77d29ce3a68d3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:08.807769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ecbf72c-a866-4284-abf9-58a157a80570 None None] MFA Rules not processed for user `c00d1d4562584ba69a7a9a5fb016b2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:08.809317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ecbf72c-a866-4284-abf9-58a157a80570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:08.829344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:08.842145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 819/1641] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:08 2022] DELETE /identity/v3/users/c73c7358f0114c0fbdb77d29ce3a68d3 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:08.857106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.859579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.859911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:08.912457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:08.914511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:08.961727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 823/1642] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:08.963665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:08.964216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:08.964711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:09.002607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] MFA Rules not processed for user `c00d1d4562584ba69a7a9a5fb016b2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:09.004311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:09.056133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 820/1643] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:09.080371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:09.087537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:09.156730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 824/1644] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:09.167529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:09.170522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:09.188915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:09.258563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d63cf2e-7b2d-4f4d-99c6-4545df542f4e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:09.293068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff37d7-bd9a-4fba-a98f-6effc0c7e279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:09.293779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff37d7-bd9a-4fba-a98f-6effc0c7e279 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:09.304954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee270d-0ce7-4129-93b0-adb21eadb06a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:09.311225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff37d7-bd9a-4fba-a98f-6effc0c7e279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:09.311570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff37d7-bd9a-4fba-a98f-6effc0c7e279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:09.312665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b56200-ffad-465e-b7ac-9a55ddf5cb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:09.313636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b56200-ffad-465e-b7ac-9a55ddf5cb20 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:09.314239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b56200-ffad-465e-b7ac-9a55ddf5cb20 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:09.314723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b56200-ffad-465e-b7ac-9a55ddf5cb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:09.329519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b56200-ffad-465e-b7ac-9a55ddf5cb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:09.331463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b56200-ffad-465e-b7ac-9a55ddf5cb20 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:09.345542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 825/1645] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:09.357291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff37d7-bd9a-4fba-a98f-6effc0c7e279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:09.360631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff37d7-bd9a-4fba-a98f-6effc0c7e279 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:09.395092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 821/1646] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:10.666260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdaeaa-6016-4494-bb73-f003fa6b4f18 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.666811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdaeaa-6016-4494-bb73-f003fa6b4f18 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.667052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdaeaa-6016-4494-bb73-f003fa6b4f18 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.669314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 826/1647] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 25 14:40:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:10.681738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.682720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.683248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.788988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:10.790592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:10.816027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 822/1648] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:10.864930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdaeaa-6016-4494-bb73-f003fa6b4f18 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:10.866930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-77fdaeaa-6016-4494-bb73-f003fa6b4f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:10.906297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdaeaa-6016-4494-bb73-f003fa6b4f18 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:10.918115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:10.919073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:10.919354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:10.923797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] PATH_INFO: `/v3/projects/a4b0cd8531734cd6a2a9c5daba97e7e9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:10.925954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b0cd8531734cd6a2a9c5daba97e7e9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:10.933099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.042821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:11.079377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b0cd8531734cd6a2a9c5daba97e7e9', 'request_id': 'req-dc8e8f1e-9045-4587-ac33-32289fb69eba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.088528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.090501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:11.106551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1181725-11af-4062-b5ac-2a6e8c19d71f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.112615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b0cd8531734cd6a2a9c5daba97e7e9', 'request_id': 'req-dc8e8f1e-9045-4587-ac33-32289fb69eba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.144021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b0cd8531734cd6a2a9c5daba97e7e9', 'request_id': 'req-dc8e8f1e-9045-4587-ac33-32289fb69eba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.189127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] The token cache is being invalidate because project a4b0cd8531734cd6a2a9c5daba97e7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:11.189277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.191652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.197053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 827/1649] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:10 2022] DELETE /identity/v3/projects/a4b0cd8531734cd6a2a9c5daba97e7e9 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:11.201485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:11.206519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:11.307605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.308541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.308964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.309368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.316112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:11.336641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.338281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2dc124-3975-471d-8bd1-0c6434f55e07 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.358375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 823/1650] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:11.426692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e8f1e-9045-4587-ac33-32289fb69eba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:11.439433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:11.446530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:11.447091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:11.447980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] PATH_INFO: `/v3/users/3295f3a2e9d84affbdba5c0b3209ff0f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:11.450741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3295f3a2e9d84affbdba5c0b3209ff0f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:11.460718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:40:11.460718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 25 14:40:11.460718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: May 25 14:40:11.461965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:11.683279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3295f3a2e9d84affbdba5c0b3209ff0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.684248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3295f3a2e9d84affbdba5c0b3209ff0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.716964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3295f3a2e9d84affbdba5c0b3209ff0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.718081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3295f3a2e9d84affbdba5c0b3209ff0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.776184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-00b13967-5ea3-4f5d-beb9-03a1c448e680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3295f3a2e9d84affbdba5c0b3209ff0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:11.849568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 828/1651] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:11 2022] DELETE /identity/v3/users/3295f3a2e9d84affbdba5c0b3209ff0f => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:13.441840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2c1ac-5dca-4a42-9825-c1728bd89d22 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:13.442713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2c1ac-5dca-4a42-9825-c1728bd89d22 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:13.443143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2c1ac-5dca-4a42-9825-c1728bd89d22 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:13.445001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 824/1652] 10.222.0.32 () {54 vars in 960 bytes} [Wed May 25 14:40:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:13.454449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:13.455025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:13.455728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:13.481361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:13.482580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:13.604389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:13.666360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 829/1653] 10.222.0.32 () {58 vars in 1057 bytes} [Wed May 25 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:13.673590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2c1ac-5dca-4a42-9825-c1728bd89d22 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:13.701121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2c1ac-5dca-4a42-9825-c1728bd89d22 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:13.734041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:13.735036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:13.735460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:13.737642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:13.740556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:13.775161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:13.778437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e76d8-1c66-4506-ae45-2661cdbf3f90 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:13.797967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 825/1654] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:14.225599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:14.235952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:14.237086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.237627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.238027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.282457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] MFA Rules not processed for user `175d6fedca25472796216b15ecd3bc1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:14.285434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:14.436985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 826/1655] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:14.437150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ac9b9-bda0-49a7-8790-3f11dfe40fb4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:14.452649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:14.453597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:14.454500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.455033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.455197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] PATH_INFO: `/v3/projects/6620f46062ee4a658dd39d840fdfd919` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.455878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:14.456673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6620f46062ee4a658dd39d840fdfd919)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:14.458773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:14.537976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6620f46062ee4a658dd39d840fdfd919', 'request_id': 'req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:14.545052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6620f46062ee4a658dd39d840fdfd919', 'request_id': 'req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:14.550924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:14.551447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6620f46062ee4a658dd39d840fdfd919', 'request_id': 'req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:14.577666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] The token cache is being invalidate because project 6620f46062ee4a658dd39d840fdfd919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:14.577988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:14.580376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5fa09fd6-c2d9-4aa1-835c-eb5ad270bdb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:14.587542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 830/1656] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:14 2022] DELETE /identity/v3/projects/6620f46062ee4a658dd39d840fdfd919 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:14.589159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e81254-a527-48b6-9830-cd2a3bc8bc31 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.589440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e81254-a527-48b6-9830-cd2a3bc8bc31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.589730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e81254-a527-48b6-9830-cd2a3bc8bc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.648986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e2e81254-a527-48b6-9830-cd2a3bc8bc31 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:14.651947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e2e81254-a527-48b6-9830-cd2a3bc8bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:14.656269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f7378-28ca-4491-93ff-cd531bed3961 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:14.816547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 831/1657] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:14.824980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.825382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.825751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.849110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:14.851223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:14.853319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:14.854212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.854212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.854506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.860978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:14.863286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 832/1658] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:14.868480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:14.880769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c072b2-f6d2-4e69-9bb8-8aecd6af42d0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:14.889674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:14.890428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.890712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.891013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.892034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:14.892478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:14.897257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:14.901333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:14.927894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 833/1659] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:14.928828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:14.944627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c345c5-8292-4975-a94c-f32fd3f580df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:14.955812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:14.957688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:14.959926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:14.963009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:14.963461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:14.966090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:14.967285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:14.969621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:14.984070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 827/1660] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:15.033311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 834/1661] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:14 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:15.034912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:15.048916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cc552-2050-4e5d-b85d-2367d3d3fb1c service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:15.176745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:15.177830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:15.178542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:15.179046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:15.180300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:15.184606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:15.224226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:15.234926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 828/1662] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:15 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:15.239693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:15.267211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5574c-2bac-4464-865c-ff3b48d5e7b4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:15.287883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:15.291043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:15.293645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:15.294244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:15.297163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:15.304802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:15.345788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 835/1663] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:15.352687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:15.373449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c4480-6f12-46e0-b8b6-95f722a9c697 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:15.386384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc2e94-e759-4390-a6e7-bae19de725bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:15.389910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc2e94-e759-4390-a6e7-bae19de725bc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:15.390284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc2e94-e759-4390-a6e7-bae19de725bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:15.390805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc2e94-e759-4390-a6e7-bae19de725bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:15.392483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc2e94-e759-4390-a6e7-bae19de725bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:15.405470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc2e94-e759-4390-a6e7-bae19de725bc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:15.434727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 829/1664] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:15.442043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:15.454855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69c0d6-537f-4680-8fd1-a2edeeebd55a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:15.461401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:15.462430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:15.462718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:15.463026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] PATH_INFO: `/v3/projects/4f84609d6e874f80a3a167c6cd8b5357/users/595056feab11406fb92a6ef4bc19cd40/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:15.524943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f84609d6e874f80a3a167c6cd8b5357, user_id=595056feab11406fb92a6ef4bc19cd40, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:15.533650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:15.598887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 836/1665] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:15 2022] PUT /identity/v3/projects/4f84609d6e874f80a3a167c6cd8b5357/users/595056feab11406fb92a6ef4bc19cd40/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:15.613290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:15.613803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:15.613896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:15.688840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] MFA Rules not processed for user `595056feab11406fb92a6ef4bc19cd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:15.692003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:15.850934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 830/1666] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:15.855456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:16.010385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9520ae-8261-4772-bb56-c7570d5b344e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:16.015849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:16.016575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.016901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.017208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.017897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:16.019906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:16.066436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 837/1667] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:15 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:16.069195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:16.085225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9be5a-3794-4c3a-9cee-f1823103e9a3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:16.200724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:16.203628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.204243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.204710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.205768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:16.209440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:16.244235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 831/1668] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:16 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:16.244949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:16.258288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afb762-0a09-4d91-9d58-9b0bcf4c08f8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:16.282282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:16.283633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.284111 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.285412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.286413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:16.294253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:16.322071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:16.322998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 838/1669] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:40:16.347786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc3c74-8e86-4ee8-bb4d-c567747d4793 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:16.376403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:16.377481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.377770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.378045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] PATH_INFO: `/v3/projects/57761cc59376409ea8c05743b2ae545c/users/ffcd90ce08404db1b90897005c579183/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.432284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=57761cc59376409ea8c05743b2ae545c, user_id=ffcd90ce08404db1b90897005c579183, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:16.435223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:16.472549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:16.478943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 832/1670] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:16 2022] PUT /identity/v3/projects/57761cc59376409ea8c05743b2ae545c/users/ffcd90ce08404db1b90897005c579183/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:16.495145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc2433-3b7f-44cd-8180-58617042721c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:16.651636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32575da-69f6-488f-a955-2979c11e5fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:16.652374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32575da-69f6-488f-a955-2979c11e5fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.652789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32575da-69f6-488f-a955-2979c11e5fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.653064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32575da-69f6-488f-a955-2979c11e5fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.654158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32575da-69f6-488f-a955-2979c11e5fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:16.656851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32575da-69f6-488f-a955-2979c11e5fd8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:16.686182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 839/1671] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:16.690785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:16.710625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75b026-30b9-4eed-80bd-9cade77f70bd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:16.720677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:16.721947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.722551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.723020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] PATH_INFO: `/v3/projects/57761cc59376409ea8c05743b2ae545c/users/ffcd90ce08404db1b90897005c579183/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.726023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57761cc59376409ea8c05743b2ae545c, user_id=ffcd90ce08404db1b90897005c579183, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:16.733215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:16.806939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:16.807355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 833/1672] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:16 2022] PUT /identity/v3/projects/57761cc59376409ea8c05743b2ae545c/users/ffcd90ce08404db1b90897005c579183/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:16.807696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:16.807975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:16.823623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:16.890966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] MFA Rules not processed for user `ffcd90ce08404db1b90897005c579183`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:16.892805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:17.085492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 840/1673] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:17.094290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:17.096858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:17.219048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:17.311326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f2d41-0c6c-41bd-bcc2-a86a56eea905 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:17.322909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:17.325924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:17.326424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:17.326584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:17.377653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:17.388881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:17.436017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 834/1674] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:17.568666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:17.569174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:17.787550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d84076-f109-4c4f-8bc0-b12b8271c5c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:17.803780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:17.819211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:17.831915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bbc0b-e1cd-4034-bbac-1ed8a3b0b0af service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:17.834404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:17.836280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:17.836852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] PATH_INFO: `/v3/users/8d8f33f0c29146c4bbbcc50c65336caa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:17.839147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8f33f0c29146c4bbbcc50c65336caa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:17.841527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:17.855832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:17.861001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:17.861290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:17.861907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] PATH_INFO: `/v3/users/595056feab11406fb92a6ef4bc19cd40` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:17.879151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=595056feab11406fb92a6ef4bc19cd40)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:17.884346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:18.006572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8f33f0c29146c4bbbcc50c65336caa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.008700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8f33f0c29146c4bbbcc50c65336caa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.032199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8f33f0c29146c4bbbcc50c65336caa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.034183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8f33f0c29146c4bbbcc50c65336caa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.060900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '595056feab11406fb92a6ef4bc19cd40'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.063689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '595056feab11406fb92a6ef4bc19cd40'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.065984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '595056feab11406fb92a6ef4bc19cd40'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.072433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a0536240-b3f5-4118-8706-316f2b7f0f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8f33f0c29146c4bbbcc50c65336caa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.098661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '595056feab11406fb92a6ef4bc19cd40'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.106173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 841/1675] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:17 2022] DELETE /identity/v3/users/8d8f33f0c29146c4bbbcc50c65336caa => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:18.109849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5473-74ee-49e0-8d24-7e32e49c1d19 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.110969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5473-74ee-49e0-8d24-7e32e49c1d19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.112674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc5473-74ee-49e0-8d24-7e32e49c1d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.138561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe500e89-9066-4649-997e-2752c49ed6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '595056feab11406fb92a6ef4bc19cd40'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:18.167397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f5bc5473-74ee-49e0-8d24-7e32e49c1d19 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:18.173969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5bc5473-74ee-49e0-8d24-7e32e49c1d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.196865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 835/1676] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:17 2022] DELETE /identity/v3/users/595056feab11406fb92a6ef4bc19cd40 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:18.201302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.201624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.203143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.209744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.285160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:18.291455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.375402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 842/1677] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:18.383417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.383706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.384080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.434497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:18.438935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.478310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 836/1678] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:40:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:18.483381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.488695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.669185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965c228-6bdc-408b-ba75-6e5da13427f1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.678166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 843/1679] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:18.679448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.680363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.680768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.681143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.700748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:18.705910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.709914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:18.712974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:18.721534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e15e1b-ad65-4913-940b-94a0a4262bb0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:18.739770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df231d-ce45-4f46-86ea-2d7e0034425b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:18.740549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df231d-ce45-4f46-86ea-2d7e0034425b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:18.740847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df231d-ce45-4f46-86ea-2d7e0034425b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:18.741128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df231d-ce45-4f46-86ea-2d7e0034425b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:18.743099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 837/1680] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:18.789761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df231d-ce45-4f46-86ea-2d7e0034425b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:18.791652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df231d-ce45-4f46-86ea-2d7e0034425b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:18.814210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:18.817133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:18.841682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 844/1681] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:18.982386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c077ad-23a6-4dee-b311-85c6b7fc2a83 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.001629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.002404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.002723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.005247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.031189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.032842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:19.057123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 838/1682] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:40:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:40:19.333479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db89645-8f6d-49ca-a7dd-8e460f2ca108 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.334043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db89645-8f6d-49ca-a7dd-8e460f2ca108 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.334881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db89645-8f6d-49ca-a7dd-8e460f2ca108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.509872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2db89645-8f6d-49ca-a7dd-8e460f2ca108 None None] MFA Rules not processed for user `175d6fedca25472796216b15ecd3bc1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:19.511754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2db89645-8f6d-49ca-a7dd-8e460f2ca108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:19.759576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 845/1683] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:19.873607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:19.876948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:19.919889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468bfb0-21c4-42d4-87aa-f6d2eaaea265 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:19.949609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:19.955376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:19.956123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:19.956656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] PATH_INFO: `/v3/users/18ff373917f2471d9e811c747a718f6a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:19.960296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ff373917f2471d9e811c747a718f6a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:19.969574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:20.170016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ff373917f2471d9e811c747a718f6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:20.173457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ff373917f2471d9e811c747a718f6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:20.174451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ff373917f2471d9e811c747a718f6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:20.208148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ff373917f2471d9e811c747a718f6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:20.239427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ff373917f2471d9e811c747a718f6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:20.266998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 839/1684] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:19 2022] DELETE /identity/v3/users/18ff373917f2471d9e811c747a718f6a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:20.273702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd469c78-be10-4798-b88a-169ee8dcb00d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:20.273994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd469c78-be10-4798-b88a-169ee8dcb00d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:20.274277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd469c78-be10-4798-b88a-169ee8dcb00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:20.323377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dd469c78-be10-4798-b88a-169ee8dcb00d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:20.325735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dd469c78-be10-4798-b88a-169ee8dcb00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:20.488445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:20.551209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 846/1685] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:20.937304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:20.940208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:21.131995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1b69c-9dd3-48dc-97c8-7682b7ca80fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:21.154646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31999ac3-f4fe-445c-8752-0140e331f777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:21.155445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31999ac3-f4fe-445c-8752-0140e331f777 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:21.155723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31999ac3-f4fe-445c-8752-0140e331f777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:21.156012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31999ac3-f4fe-445c-8752-0140e331f777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:21.194096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31999ac3-f4fe-445c-8752-0140e331f777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:21.195869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31999ac3-f4fe-445c-8752-0140e331f777 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:21.200540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:21.231217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 840/1686] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:22.004924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:22.005875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:22.006290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:22.068531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] MFA Rules not processed for user `73d50022c5f94444afc67ea810e4d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:22.070037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:22.283161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 847/1687] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:22.295507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f942e62-a816-4e76-afbb-bed141b6c196 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:22.296341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f942e62-a816-4e76-afbb-bed141b6c196 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:22.296341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f942e62-a816-4e76-afbb-bed141b6c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:22.362500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5f942e62-a816-4e76-afbb-bed141b6c196 None None] MFA Rules not processed for user `73d50022c5f94444afc67ea810e4d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:22.364617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5f942e62-a816-4e76-afbb-bed141b6c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:22.410058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 841/1688] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:22.423686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:22.425549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:22.635143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:22.635688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:22.636075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:22.691596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:22.698988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:22.733751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 842/1689] 10.222.0.75 () {58 vars in 1015 bytes} [Wed May 25 14:40:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:22.759142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42815f0-64f7-4d3d-aab7-009c485cec6b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:22.770561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:22.771462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:22.771850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:22.772233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:22.810681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:22.813331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:22.837408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 848/1690] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:22.968201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:22.969879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.003828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0757dd-68fe-4c84-a5f6-5eb0cf8ba1f9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:23.010222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.024061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53377617-3934-4a9d-9c4d-f168e1002955 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:23.045429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:23.046684 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d347c4-ede7-4cc4-9e47-9a55e0fdd51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:23.047973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.049354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d347c4-ede7-4cc4-9e47-9a55e0fdd51e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.049979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.050472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d347c4-ede7-4cc4-9e47-9a55e0fdd51e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.051081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d347c4-ede7-4cc4-9e47-9a55e0fdd51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.051523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] PATH_INFO: `/v3/projects/4f84609d6e874f80a3a167c6cd8b5357` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.054166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f84609d6e874f80a3a167c6cd8b5357)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:23.057218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 14:40:23.057218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 25 14:40:23.057218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: May 25 14:40:23.068667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:23.085657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d347c4-ede7-4cc4-9e47-9a55e0fdd51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:23.088020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d347c4-ede7-4cc4-9e47-9a55e0fdd51e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:23.115113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 849/1691] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:23.125503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c17cf-4fe7-4172-a7a7-7a2059a7e144 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.130157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c17cf-4fe7-4172-a7a7-7a2059a7e144 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.130605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c17cf-4fe7-4172-a7a7-7a2059a7e144 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.132289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 850/1692] 10.222.0.32 () {54 vars in 912 bytes} [Wed May 25 14:40:23 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:23.139182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.140565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.141964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.194818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:23.200139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.250292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f84609d6e874f80a3a167c6cd8b5357', 'request_id': 'req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.289121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f84609d6e874f80a3a167c6cd8b5357', 'request_id': 'req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.306104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f84609d6e874f80a3a167c6cd8b5357', 'request_id': 'req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.309280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:23.344385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 851/1693] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:40:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:23.352076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] The token cache is being invalidate because project 4f84609d6e874f80a3a167c6cd8b5357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:23.353771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.358284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.362566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 843/1694] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:22 2022] DELETE /identity/v3/projects/4f84609d6e874f80a3a167c6cd8b5357 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:23.368261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.489227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9066d3-062c-4e75-bea9-4b14b2d4c3df None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:23.515644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:23.516578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.516857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.517127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] PATH_INFO: `/v3/users/ffcd90ce08404db1b90897005c579183` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.523807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffcd90ce08404db1b90897005c579183)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:23.525914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:23.630976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffcd90ce08404db1b90897005c579183'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.632456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffcd90ce08404db1b90897005c579183'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.643376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffcd90ce08404db1b90897005c579183'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.646011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:23.648024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.653096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffcd90ce08404db1b90897005c579183'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.671184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4ee28-4c40-41d7-ae80-2c9e1c482b6d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:23.687298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:23.689108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:23.690103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:23.692070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:23.696130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:23.698785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffcd90ce08404db1b90897005c579183'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:23.727309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:23.735794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 852/1695] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:23 2022] DELETE /identity/v3/users/ffcd90ce08404db1b90897005c579183 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:23.737800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:23.757028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 844/1696] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:25.302908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.304772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:25.718102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791df77-99a6-458d-b92a-2e57d88b2d53 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:25.739273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:25.740170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:25.740470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:25.740721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] PATH_INFO: `/v3/projects/ff4a889141fb4ed7bf679ca8c4623d16` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:25.745160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4a889141fb4ed7bf679ca8c4623d16)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:25.755471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:25.846604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:25.961265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:25.962976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:26.018006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abc7b3-eff9-4335-a56d-e822b8053044 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.046518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4a889141fb4ed7bf679ca8c4623d16', 'request_id': 'req-7bd66d29-c440-454c-8eba-f4861f8f444c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.087568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4a889141fb4ed7bf679ca8c4623d16', 'request_id': 'req-7bd66d29-c440-454c-8eba-f4861f8f444c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.106022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4a889141fb4ed7bf679ca8c4623d16', 'request_id': 'req-7bd66d29-c440-454c-8eba-f4861f8f444c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.154171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] The token cache is being invalidate because project ff4a889141fb4ed7bf679ca8c4623d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:26.155694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.157651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd66d29-c440-454c-8eba-f4861f8f444c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.163870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 853/1697] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:25 2022] DELETE /identity/v3/projects/ff4a889141fb4ed7bf679ca8c4623d16 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:26.182410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cac8a-7875-487b-9834-a587b874bbce None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.183847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cac8a-7875-487b-9834-a587b874bbce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.184235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cac8a-7875-487b-9834-a587b874bbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.253866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-891cac8a-7875-487b-9834-a587b874bbce None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:26.260809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-891cac8a-7875-487b-9834-a587b874bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:26.370304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.371260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.371536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.371808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] PATH_INFO: `/v3/projects/4753380d566d4307bf8d1bc0eeca8c15` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.379086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4753380d566d4307bf8d1bc0eeca8c15)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.381087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.395078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 854/1698] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:26.399823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.401857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.404300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.474639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:26.480514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:26.515698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4753380d566d4307bf8d1bc0eeca8c15', 'request_id': 'req-ca25ba43-4db0-496b-8330-a05da6eb52f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.544028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4753380d566d4307bf8d1bc0eeca8c15', 'request_id': 'req-ca25ba43-4db0-496b-8330-a05da6eb52f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.550077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:26.561473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4753380d566d4307bf8d1bc0eeca8c15', 'request_id': 'req-ca25ba43-4db0-496b-8330-a05da6eb52f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.596452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] The token cache is being invalidate because project 4753380d566d4307bf8d1bc0eeca8c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:26.596815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.601255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca25ba43-4db0-496b-8330-a05da6eb52f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:26.608825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 845/1699] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:25 2022] DELETE /identity/v3/projects/4753380d566d4307bf8d1bc0eeca8c15 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:26.610946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443e5b-5346-4826-af06-7f9cf7ce713e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.611235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443e5b-5346-4826-af06-7f9cf7ce713e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.611512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a443e5b-5346-4826-af06-7f9cf7ce713e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.657315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 855/1700] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:26.666712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.690571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7a443e5b-5346-4826-af06-7f9cf7ce713e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:26.694418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7a443e5b-5346-4826-af06-7f9cf7ce713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:26.698779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1f718-c1d9-49aa-b8d0-386098dc58a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.705470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.706550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.707160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.707160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.708807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:26.709243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.712200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.733044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 856/1701] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:26.737951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 846/1702] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:26.738449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.750803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.751083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.751362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.756947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036c995-b02d-4f76-8c34-8b88f9e31bab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:26.763241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:26.765071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:26.765822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:26.766397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:26.772340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:26.772340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:26.873359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 857/1703] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:26.884564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:26.913463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:27.020473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadddfcb-58c7-47ec-8f99-586a317b3768 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.036675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.038603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.040801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.041924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.045497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.051784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.110009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 858/1704] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:27.114550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.144760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55025be-749a-480c-8a2e-baaae9b79b76 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.165355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.166108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.166391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.171914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.172769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.176380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.233083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 859/1705] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:27.236024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.236409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:27.241250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:27.266266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3ab5b-68b3-4b96-886c-729c0741c823 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.282070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.282806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.283082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.286017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.287016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.294553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.321804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 847/1706] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 568 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:27.323917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.347465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e546cd-ab95-413c-a5a0-a4c87d1562d3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.360546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.363123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.363414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.363689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.364473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:27.364779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.372496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.383896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 860/1707] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:27.387830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.434319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 848/1708] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:27.439193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.452237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229d4f3-2a84-4f51-bd8f-a49c2336a4c4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.473948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.477162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59774db-279b-427f-97db-2de5d5ef98cd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.478953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.479246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.479523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] PATH_INFO: `/v3/projects/155ebb991459454686c07cbc8adc3b69/users/c64343bf0fd44781a687fe0fa866ff43/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.489958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.491534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.492333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.492875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.494087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.502857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.550097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155ebb991459454686c07cbc8adc3b69, user_id=c64343bf0fd44781a687fe0fa866ff43, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.558174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:27.592371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 861/1709] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:27 2022] PUT /identity/v3/projects/155ebb991459454686c07cbc8adc3b69/users/c64343bf0fd44781a687fe0fa866ff43/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:27.599952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.605272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:27.616254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 849/1710] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:27 2022] POST /identity/v3/projects => generated 405 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:27.621354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:27.865708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9bf56-453e-4c13-bc4d-b84ce8518324 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.900997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8e76a-7fbb-40c0-b133-f58db8ab50ab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:27.913025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060acb4b-71c9-4896-a3f0-9673b0dd69e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:27.914329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060acb4b-71c9-4896-a3f0-9673b0dd69e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:27.916000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060acb4b-71c9-4896-a3f0-9673b0dd69e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:27.917969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060acb4b-71c9-4896-a3f0-9673b0dd69e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:27.919621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060acb4b-71c9-4896-a3f0-9673b0dd69e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:27.939151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060acb4b-71c9-4896-a3f0-9673b0dd69e7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.023451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 850/1711] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:27 2022] POST /identity/v3/users => generated 303 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:28.023451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.026787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.027388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.120799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] MFA Rules not processed for user `c64343bf0fd44781a687fe0fa866ff43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:28.122506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:28.190567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.193524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.193524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.193813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] PATH_INFO: `/v3/projects/57761cc59376409ea8c05743b2ae545c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.199644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57761cc59376409ea8c05743b2ae545c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.208577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.213263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:28.336976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 851/1712] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:28.339313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.365566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eab72-d32d-451c-94f3-9fa4ac241fae None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.428070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:28.438236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57761cc59376409ea8c05743b2ae545c', 'request_id': 'req-3a6b14f9-df29-4e42-b815-07c3ca8973c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:28.483129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57761cc59376409ea8c05743b2ae545c', 'request_id': 'req-3a6b14f9-df29-4e42-b815-07c3ca8973c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:28.523518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57761cc59376409ea8c05743b2ae545c', 'request_id': 'req-3a6b14f9-df29-4e42-b815-07c3ca8973c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:28.554494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9958360-68cd-464c-8229-851743796391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.555401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9958360-68cd-464c-8229-851743796391 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.555790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9958360-68cd-464c-8229-851743796391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.556169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9958360-68cd-464c-8229-851743796391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.557035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9958360-68cd-464c-8229-851743796391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.558845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9958360-68cd-464c-8229-851743796391 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.607491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 852/1713] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:28.612400 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0ec0c-0625-4959-a332-9ea60f80d604 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.618271 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0ec0c-0625-4959-a332-9ea60f80d604 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.618776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0ec0c-0625-4959-a332-9ea60f80d604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.653958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] The token cache is being invalidate because project 57761cc59376409ea8c05743b2ae545c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:28.659729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:28.665750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:28.683879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 862/1714] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:27 2022] DELETE /identity/v3/projects/57761cc59376409ea8c05743b2ae545c => generated 0 bytes in 1091 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:28.693651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2aa0ec0c-0625-4959-a332-9ea60f80d604 None None] MFA Rules not processed for user `73d50022c5f94444afc67ea810e4d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:28.693859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.700297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2aa0ec0c-0625-4959-a332-9ea60f80d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:28.701330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:28.797453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b14f9-df29-4e42-b815-07c3ca8973c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:28.818110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:28.821700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.822169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.824842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:28.826215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:28.837593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:28.929579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 863/1715] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:28 2022] POST /identity/v3/projects => generated 367 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:28.929579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:28.936986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:28.995813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 853/1716] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:28.998448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:28.999029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:28.999682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.041181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:29.042452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:29.167160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc09f3e-aebc-4bb5-811f-bd5ec37050e7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.223786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.225083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.237509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.237965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.238901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.242299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.255765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 864/1717] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:29.256959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.271559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec2990-ee4b-42ee-9e7e-a7521b6c0b48 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.281017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2d734-27b1-4210-b6c5-3f576e28bf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.281515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2d734-27b1-4210-b6c5-3f576e28bf08 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.281937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2d734-27b1-4210-b6c5-3f576e28bf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.282347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2d734-27b1-4210-b6c5-3f576e28bf08 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.283296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2d734-27b1-4210-b6c5-3f576e28bf08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.286665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2d734-27b1-4210-b6c5-3f576e28bf08 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.307654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 854/1718] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:29.310028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.334488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 865/1719] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:29 2022] POST /identity/v3/users => generated 290 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:29.337911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.338374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.338884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.351522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609937-230b-4de3-9243-78f86713cc74 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.370494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.371427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.371709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.372199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] PATH_INFO: `/v3/projects/6e4c207993954b828bc4ffb06b456952/users/24dd7b07950541d3b0307ab268ef87a0/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.398683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:29.400209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:29.428751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 866/1720] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:29.428751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.429063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4c207993954b828bc4ffb06b456952, user_id=24dd7b07950541d3b0307ab268ef87a0, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.435475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.453330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fedd77-8a08-4cad-85e5-8659fdb91515 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.467566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb80573-9d6f-4319-8e2f-9359ec93e830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.470241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80573-9d6f-4319-8e2f-9359ec93e830 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.470872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80573-9d6f-4319-8e2f-9359ec93e830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.471463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80573-9d6f-4319-8e2f-9359ec93e830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.472877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80573-9d6f-4319-8e2f-9359ec93e830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.481217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80573-9d6f-4319-8e2f-9359ec93e830 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.483007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 855/1721] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:29 2022] PUT /identity/v3/projects/6e4c207993954b828bc4ffb06b456952/users/24dd7b07950541d3b0307ab268ef87a0/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:29.483930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.560659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 867/1722] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:29.562228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.562519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.562878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.630453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:29.631960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:29.677357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:29.752463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:29.797119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80116dab-f808-4525-9d19-1ba43b28e1d6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.801913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.810311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.810855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.811150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.812235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:29.813119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.816845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.835025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 856/1723] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:29.835914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.861824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfc390-4546-49a8-837f-8ac52380a58e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:29.877391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690183b-72f0-4b5a-b714-43d92d013bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:29.878517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690183b-72f0-4b5a-b714-43d92d013bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.878731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690183b-72f0-4b5a-b714-43d92d013bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.878928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690183b-72f0-4b5a-b714-43d92d013bb8 admin admin] PATH_INFO: `/v3/projects/f89aea4c147c4df69e4a29357cc2a998/users/79a2218682c84ddda521885588d4cc60/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.929551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690183b-72f0-4b5a-b714-43d92d013bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f89aea4c147c4df69e4a29357cc2a998, user_id=79a2218682c84ddda521885588d4cc60, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:29.937953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690183b-72f0-4b5a-b714-43d92d013bb8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:29.975094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 857/1724] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:29 2022] PUT /identity/v3/projects/f89aea4c147c4df69e4a29357cc2a998/users/79a2218682c84ddda521885588d4cc60/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:29.977174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f9015-52ae-4626-b5cf-236d9f7ad46c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:29.979582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f9015-52ae-4626-b5cf-236d9f7ad46c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:29.980279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f9015-52ae-4626-b5cf-236d9f7ad46c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:29.985495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 858/1725] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:40:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:29.986727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 868/1726] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:29.987987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f9015-52ae-4626-b5cf-236d9f7ad46c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:29.988350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:30.037937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b36bf8-ebf2-4499-a167-28fe8d1a4ac6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:30.174785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f9015-52ae-4626-b5cf-236d9f7ad46c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:30.196054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:30.196839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.197084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.197474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.198300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:30.200436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:30.252202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00a88d-dbe5-46ec-8437-d9b4a9e4b498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:30.252939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 859/1727] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:29 2022] POST /identity/v3/projects => generated 381 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:30.253375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a88d-dbe5-46ec-8437-d9b4a9e4b498 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.253891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a88d-dbe5-46ec-8437-d9b4a9e4b498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.254386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a88d-dbe5-46ec-8437-d9b4a9e4b498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.255796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a88d-dbe5-46ec-8437-d9b4a9e4b498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:30.257614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:30.259127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a88d-dbe5-46ec-8437-d9b4a9e4b498 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:30.260891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:30.325330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 869/1728] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:30.327341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.327814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.328230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.535911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] MFA Rules not processed for user `2401371a775743c29c16406cb9df5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:30.537199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:30.633413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e416e57-02fe-4217-a12b-975f74f19fda admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:30.646196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:30.646916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:30.647175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:30.647446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:30.881538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:30.911798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 870/1729] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:30.912407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.007953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.009452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.099887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 860/1730] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 846 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:31.110148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.138865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b564d-6830-44b9-bf67-0d874d69f7fa service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.161871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df7edf-fe6b-49ca-afd1-d9543f4b5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.162814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df7edf-fe6b-49ca-afd1-d9543f4b5b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.166711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df7edf-fe6b-49ca-afd1-d9543f4b5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.167615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df7edf-fe6b-49ca-afd1-d9543f4b5b2a admin admin] PATH_INFO: `/v3/projects/f89aea4c147c4df69e4a29357cc2a998/users/79a2218682c84ddda521885588d4cc60/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.176302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df7edf-fe6b-49ca-afd1-d9543f4b5b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f89aea4c147c4df69e4a29357cc2a998, user_id=79a2218682c84ddda521885588d4cc60, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.184802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1df7edf-fe6b-49ca-afd1-d9543f4b5b2a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.227829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:31.230258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 861/1731] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:31 2022] PUT /identity/v3/projects/f89aea4c147c4df69e4a29357cc2a998/users/79a2218682c84ddda521885588d4cc60/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:31.231843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.234532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.234861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.268379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469270f3-e266-401d-85f9-3c11c5c7f4b0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.284302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:31.292279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:31.441551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 862/1732] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:40:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:31.442263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.460831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:31.534964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fae3bd-6c2e-4e73-9a4f-c1ed484bf3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.535695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fae3bd-6c2e-4e73-9a4f-c1ed484bf3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.535980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fae3bd-6c2e-4e73-9a4f-c1ed484bf3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.536467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fae3bd-6c2e-4e73-9a4f-c1ed484bf3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.541290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bcd50-17ce-4436-8ceb-593cc032ddec None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.541446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fae3bd-6c2e-4e73-9a4f-c1ed484bf3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.549876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fae3bd-6c2e-4e73-9a4f-c1ed484bf3eb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.558822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.559701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.560243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.560987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.616924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 871/1733] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:30 2022] POST /identity/v3/users => generated 291 bytes in 702 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:31.616924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.616924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.616924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.709812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] MFA Rules not processed for user `79a2218682c84ddda521885588d4cc60`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:31.711985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:31.814902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.820478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.869793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 863/1734] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:31.876416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.918272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bada21e-c9e0-40f5-b595-a2f858a229df service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:31.942484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:31.946222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:31.948251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:31.950558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:31.951923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:31.961661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:31.974209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 872/1735] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:31.983901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:31.988785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:32.018435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 864/1736] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:32.019505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:32.049088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8ea0-e5fd-40b4-9ff0-96e5dc1021f2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.055202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.055910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.059847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.060247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.061017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:32.062715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:32.100337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 865/1737] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:32.104454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:32.137324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647d0a-6123-4a06-8711-a8f16cac8058 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.159163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91151e59-7277-49e7-b87a-ac375cc5d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.167593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151e59-7277-49e7-b87a-ac375cc5d6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.168140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151e59-7277-49e7-b87a-ac375cc5d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.168552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91151e59-7277-49e7-b87a-ac375cc5d6c9 admin admin] PATH_INFO: `/v3/projects/514aee8945e747cfa74a6a08f74dd05b/users/0b5eadc470864ec3a7832696837d0437/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.214258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91151e59-7277-49e7-b87a-ac375cc5d6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=514aee8945e747cfa74a6a08f74dd05b, user_id=0b5eadc470864ec3a7832696837d0437, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:32.216977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb114-6379-4b07-b0a7-00ba40ffff0d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.221494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91151e59-7277-49e7-b87a-ac375cc5d6c9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:32.230666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.234191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.234969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.235778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.252742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 866/1738] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:32 2022] PUT /identity/v3/projects/514aee8945e747cfa74a6a08f74dd05b/users/0b5eadc470864ec3a7832696837d0437/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:32.266865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.267338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.267755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.287444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:32.293195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:32.296632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:32.321104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] MFA Rules not processed for user `0b5eadc470864ec3a7832696837d0437`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:32.328887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:32.350863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 873/1739] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:32.605540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:32.605826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 867/1740] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:32.638969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fbba7-720c-42bf-ad66-f9007454747b service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.639160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:32.715987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:32.877169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fabf8-fd35-42c9-83fc-70b556669048 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:32.896860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.897730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.898233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.898629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.899159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:32.899512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:32.905362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:32.905723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:32.905917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:32.912208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:32.916040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:32.983964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 874/1741] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:32 2022] POST /identity/v3/projects => generated 381 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:32.987712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.172708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ccbd7-4def-4698-9ca6-602df0dc3c4f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.209089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.209089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.209089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.209089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.209503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.209503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.262411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.270528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.308690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 868/1742] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:33.325137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 875/1743] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:32 2022] POST /identity/v3/users => generated 297 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:33.329726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.368972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cb895-3182-465b-b17d-3e299f003f03 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.384147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.385081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.385593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.385902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.390932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.392611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.428162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 869/1744] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:33.432104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.457486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85386b-2892-4d07-aa75-42aefea3358e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.469308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.470246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.470527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.470923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] PATH_INFO: `/v3/projects/cbb46488403a41fb96f1545a8427c864/users/2a9a406df5d242c2b3518e0023f2247c/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.513860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb46488403a41fb96f1545a8427c864, user_id=2a9a406df5d242c2b3518e0023f2247c, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.520813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.568428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 876/1745] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:33 2022] PUT /identity/v3/projects/cbb46488403a41fb96f1545a8427c864/users/2a9a406df5d242c2b3518e0023f2247c/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:33.573550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.599794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7dac-8e09-4bcc-b8fc-0025300608b2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.730048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:33.753030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd07b-d524-4dbb-88a7-8c1a06823fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.753784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd07b-d524-4dbb-88a7-8c1a06823fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.754071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd07b-d524-4dbb-88a7-8c1a06823fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.754347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd07b-d524-4dbb-88a7-8c1a06823fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.755432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcd07b-d524-4dbb-88a7-8c1a06823fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.757152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcd07b-d524-4dbb-88a7-8c1a06823fa7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.775352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:33.776227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 870/1746] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:33.834063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c76be-703b-4346-821d-0a782e1c4d5e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:33.879739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:33.880966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.881405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.881813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] PATH_INFO: `/v3/projects/cbb46488403a41fb96f1545a8427c864/users/2a9a406df5d242c2b3518e0023f2247c/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:33.890340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb46488403a41fb96f1545a8427c864, user_id=2a9a406df5d242c2b3518e0023f2247c, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:33.903305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:33.953387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 877/1747] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:33 2022] PUT /identity/v3/projects/cbb46488403a41fb96f1545a8427c864/users/2a9a406df5d242c2b3518e0023f2247c/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:33.973948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4800337-9773-4d50-ae5c-e5a4c61c8203 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:33.977071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4800337-9773-4d50-ae5c-e5a4c61c8203 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:33.977611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4800337-9773-4d50-ae5c-e5a4c61c8203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.165832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b4800337-9773-4d50-ae5c-e5a4c61c8203 None None] MFA Rules not processed for user `2a9a406df5d242c2b3518e0023f2247c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:34.174600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4800337-9773-4d50-ae5c-e5a4c61c8203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:34.421688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 871/1748] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:34.433921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.526803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.527688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.528644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.598001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] MFA Rules not processed for user `2d2256afe4094555b206050fb1e09365`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:34.603131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:34.630615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60b3e2-0c61-4f60-8ad2-1ca769da0ece admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:34.662055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:34.662740 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:34.663043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:34.663317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:34.702863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:34.704344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:34.735095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 878/1749] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:34.741129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:34.748384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:34.768396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:34.887510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 872/1750] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:35.082831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d40ba8-54f4-47da-98eb-5775f012d626 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.107582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecffd4d-f1b0-4f8c-8964-ae1854199aa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.108915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecffd4d-f1b0-4f8c-8964-ae1854199aa1 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.109984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecffd4d-f1b0-4f8c-8964-ae1854199aa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.111526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecffd4d-f1b0-4f8c-8964-ae1854199aa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.136829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecffd4d-f1b0-4f8c-8964-ae1854199aa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:35.148861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecffd4d-f1b0-4f8c-8964-ae1854199aa1 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:35.194966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 879/1751] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:35.753621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:35.916319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:35.944346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ec8b5-600b-49e1-a4e3-5b2989f02432 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:35.956655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:35.957876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:35.958619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:35.960060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] PATH_INFO: `/v3/users/0b5eadc470864ec3a7832696837d0437` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:35.963483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5eadc470864ec3a7832696837d0437)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:35.968463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.084474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5eadc470864ec3a7832696837d0437'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:36.088511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5eadc470864ec3a7832696837d0437'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:36.089315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5eadc470864ec3a7832696837d0437'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:36.117351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5eadc470864ec3a7832696837d0437'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:36.141863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5eadc470864ec3a7832696837d0437'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:36.192712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 873/1752] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:35 2022] DELETE /identity/v3/users/0b5eadc470864ec3a7832696837d0437 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:36.194881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.195420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.195502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.228528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:36.230141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:36.300612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.302339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:36.497559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 880/1753] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:36.518843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d17d4e-c422-473c-92b6-8ccd29c184b0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.525235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.526246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.527299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:36.529954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:36.534071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.534323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.540476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:36.690640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732072f-bf63-4969-b1a5-3a900982c9d6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:36.703658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:36.705623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:36.705911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:36.708058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:36.731743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.737756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:36.738457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa8125-923b-4fdb-9170-209e58da7616 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.743627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:36.769984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 874/1754] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:36.771003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 881/1755] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:37.868390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:37.869320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:37.870231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:37.924663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:37.956480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] MFA Rules not processed for user `24dd7b07950541d3b0307ab268ef87a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:37.961172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:38.090541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 875/1756] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:37 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:38.095180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.097728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:38.114987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8cd69-7126-40a1-ace9-932fb637d1a6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.122009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.123635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.124265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.124752 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.125726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.129118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.176495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 882/1757] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:38 2022] POST /identity/v3/projects => generated 403 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:38.180378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.207828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e152-1e3c-499e-93f3-bbcb10fe8e6e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.297609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.298902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.299231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.300122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.301073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.304042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.381975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 876/1758] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:38 2022] POST /identity/v3/users => generated 308 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:38.387138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.403612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18a59b-54f2-49b9-8ef3-36b9d64cac5d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.418686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.419425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.421945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.422279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.423083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.431504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.448781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:38.491119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.506306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 883/1759] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:38.509425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecd876-f84c-4a01-afe3-7e074b5dd11e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.528182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.529645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.530069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.530920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] PATH_INFO: `/v3/projects/e834bbaaea9a4355b10e06c9ececb469/users/14a2b02b3bcb47658cfb7b4d966bc731/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.571720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e834bbaaea9a4355b10e06c9ececb469, user_id=14a2b02b3bcb47658cfb7b4d966bc731, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.578551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.618397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 877/1760] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:38 2022] PUT /identity/v3/projects/e834bbaaea9a4355b10e06c9ececb469/users/14a2b02b3bcb47658cfb7b4d966bc731/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:38.624746 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.636616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48470b79-705b-4975-bbd9-6492b46db2e3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.736596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.737257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.737643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.738317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.738968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.740709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.753278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 884/1761] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:38.757403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:38.773593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf43c-fcf0-4c13-8928-e838247c3eba admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:38.787898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:38.788827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:38.789106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:38.789381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] PATH_INFO: `/v3/projects/e834bbaaea9a4355b10e06c9ececb469/users/14a2b02b3bcb47658cfb7b4d966bc731/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:38.798445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e834bbaaea9a4355b10e06c9ececb469, user_id=14a2b02b3bcb47658cfb7b4d966bc731, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:38.802398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:38.828137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 878/1762] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:38 2022] PUT /identity/v3/projects/e834bbaaea9a4355b10e06c9ececb469/users/14a2b02b3bcb47658cfb7b4d966bc731/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:39.307867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:39.309654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:39.330998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f0a9a-d754-439c-89dc-d714b1364fef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:39.469721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:39.488841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:39.489275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:39.490247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] PATH_INFO: `/v3/projects/514aee8945e747cfa74a6a08f74dd05b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:39.493169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=514aee8945e747cfa74a6a08f74dd05b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:39.496859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:39.663951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514aee8945e747cfa74a6a08f74dd05b', 'request_id': 'req-0db285d5-2ebc-4dce-8b62-f840c15da306'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:39.687932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514aee8945e747cfa74a6a08f74dd05b', 'request_id': 'req-0db285d5-2ebc-4dce-8b62-f840c15da306'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:39.699569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514aee8945e747cfa74a6a08f74dd05b', 'request_id': 'req-0db285d5-2ebc-4dce-8b62-f840c15da306'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:39.743056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] The token cache is being invalidate because project 514aee8945e747cfa74a6a08f74dd05b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:39.743358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:39.745722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0db285d5-2ebc-4dce-8b62-f840c15da306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:39.753749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 885/1763] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:39 2022] DELETE /identity/v3/projects/514aee8945e747cfa74a6a08f74dd05b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:39.754699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:39.759114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:39.822931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd5dffd-321e-4941-aac0-dc20009a15f0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:39.939818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:39.962886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:39.964032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:39.964345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:39.966829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] PATH_INFO: `/v3/users/2a9a406df5d242c2b3518e0023f2247c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:39.968233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a9a406df5d242c2b3518e0023f2247c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:39.970378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:40.128495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a9a406df5d242c2b3518e0023f2247c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:40.129372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a9a406df5d242c2b3518e0023f2247c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:40.130445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a9a406df5d242c2b3518e0023f2247c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:40.157409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a9a406df5d242c2b3518e0023f2247c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:40.168451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314890-a456-4a62-854f-f21c4017a7d8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.168919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314890-a456-4a62-854f-f21c4017a7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.169030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a9a406df5d242c2b3518e0023f2247c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:40.169577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4314890-a456-4a62-854f-f21c4017a7d8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.171469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 886/1764] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:40:40.207458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 879/1765] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:39 2022] DELETE /identity/v3/users/2a9a406df5d242c2b3518e0023f2247c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:40.222031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853562-4626-477d-ac62-6eb03b709e71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.222545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853562-4626-477d-ac62-6eb03b709e71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.223018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10853562-4626-477d-ac62-6eb03b709e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.271380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10853562-4626-477d-ac62-6eb03b709e71 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:40.272778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10853562-4626-477d-ac62-6eb03b709e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:40.312677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:40.522389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 887/1766] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:40:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:40.845438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:40.847543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:40.867028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b37d0b-7aa6-4aa3-80ce-5a7d59973158 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:40.881231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:40.882253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:40.882585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:40.882897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:40.931414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:40.933051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:40.945188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 880/1767] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:41.102600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b487cf-8c5f-4832-bb44-55e763e8619a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.102920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b487cf-8c5f-4832-bb44-55e763e8619a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.103190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b487cf-8c5f-4832-bb44-55e763e8619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.125321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-13b487cf-8c5f-4832-bb44-55e763e8619a None None] MFA Rules not processed for user `2401371a775743c29c16406cb9df5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:41.126712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13b487cf-8c5f-4832-bb44-55e763e8619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.202916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 888/1768] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:41.435959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:41.437741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.546685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08dfb7-7c9e-45da-8244-4e4e05fda572 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:41.552914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:41.554081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.554464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.554867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] PATH_INFO: `/v3/projects/cbb46488403a41fb96f1545a8427c864` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.556170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb46488403a41fb96f1545a8427c864)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:41.558234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:41.632206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb46488403a41fb96f1545a8427c864', 'request_id': 'req-99ede9ff-9d27-41ce-b71a-0f9805889538'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:41.644874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb46488403a41fb96f1545a8427c864', 'request_id': 'req-99ede9ff-9d27-41ce-b71a-0f9805889538'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:41.654278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:41.662723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb46488403a41fb96f1545a8427c864', 'request_id': 'req-99ede9ff-9d27-41ce-b71a-0f9805889538'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:41.684532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa7d01-c973-4b02-a5bc-43cad4129cb1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.686064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa7d01-c973-4b02-a5bc-43cad4129cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.686585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa7d01-c973-4b02-a5bc-43cad4129cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.699164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] The token cache is being invalidate because project cbb46488403a41fb96f1545a8427c864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:41.699464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:41.701224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-99ede9ff-9d27-41ce-b71a-0f9805889538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:41.711611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 881/1769] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:41 2022] DELETE /identity/v3/projects/cbb46488403a41fb96f1545a8427c864 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:41.713819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87ed9-8da8-4c05-9ecd-4d11d21efa1b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.714117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87ed9-8da8-4c05-9ecd-4d11d21efa1b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.714566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3efa7d01-c973-4b02-a5bc-43cad4129cb1 None None] MFA Rules not processed for user `2d2256afe4094555b206050fb1e09365`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:41.715904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3efa7d01-c973-4b02-a5bc-43cad4129cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.716504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87ed9-8da8-4c05-9ecd-4d11d21efa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.758816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b5e87ed9-8da8-4c05-9ecd-4d11d21efa1b None None] MFA Rules not processed for user `14a2b02b3bcb47658cfb7b4d966bc731`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:41.760408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b5e87ed9-8da8-4c05-9ecd-4d11d21efa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.805066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 889/1770] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:41.812645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a003c-1980-4501-b67b-322834d44346 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.813184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a003c-1980-4501-b67b-322834d44346 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.813655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a003c-1980-4501-b67b-322834d44346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.834610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-930a003c-1980-4501-b67b-322834d44346 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:41.836943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-930a003c-1980-4501-b67b-322834d44346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.860278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 882/1771] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:41.945673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 890/1772] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:41.953883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:41.954169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:41.954444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:41.979438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:41.980755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:41.991360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 883/1773] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:41.996140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a003c-1980-4501-b67b-322834d44346 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.008945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a003c-1980-4501-b67b-322834d44346 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.014482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.015347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.015757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.016089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.017176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:42.017306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.019519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.021109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:42.035660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 891/1774] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:42.039845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.051499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c421d3-4a6a-44fa-bfed-28597d051dce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.057564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.061922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.062289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.062631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.063516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.066229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.106650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 884/1775] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:42 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:42.109650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.124681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a4c88-5575-496e-b972-1b630505c707 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.236792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.237622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.238128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.238467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.239822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.241960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.276916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 892/1776] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:42 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:42.281361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.298980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf58770-cefe-4ce4-811f-d6c6add12bfe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.307644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.308705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.309097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.309469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.310396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.314412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.333958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 885/1777] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:42.335437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.362686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc67bc-01e7-4f33-9121-4f05614f717f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.368874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f96fae-b8bf-4ce9-81cd-33b3a3d28f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.369704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f96fae-b8bf-4ce9-81cd-33b3a3d28f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.370214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f96fae-b8bf-4ce9-81cd-33b3a3d28f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.370572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f96fae-b8bf-4ce9-81cd-33b3a3d28f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.371888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f96fae-b8bf-4ce9-81cd-33b3a3d28f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.373670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f96fae-b8bf-4ce9-81cd-33b3a3d28f73 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.389571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 893/1778] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:42.391216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.403968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fc62-fb2d-45c4-bf64-06d50a22861b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.415168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.416164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.416411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.417288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] PATH_INFO: `/v3/projects/069dbbe5f20f48b480d1b8f5e93abb1f/users/be848e24e08f493ab165760a30d3d8b7/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.436903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069dbbe5f20f48b480d1b8f5e93abb1f, user_id=be848e24e08f493ab165760a30d3d8b7, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.441017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.467151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 886/1779] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:42 2022] PUT /identity/v3/projects/069dbbe5f20f48b480d1b8f5e93abb1f/users/be848e24e08f493ab165760a30d3d8b7/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:42.479705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a472-8957-48eb-9275-51c439fe7586 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.479986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a472-8957-48eb-9275-51c439fe7586 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.480267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726a472-8957-48eb-9275-51c439fe7586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.525856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f726a472-8957-48eb-9275-51c439fe7586 None None] MFA Rules not processed for user `be848e24e08f493ab165760a30d3d8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:42.533314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f726a472-8957-48eb-9275-51c439fe7586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:42.647718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 894/1780] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:42.656216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:42.658269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:42.730318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d2d94-63af-42b2-a963-755173b45b98 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:42.734332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c7b48-8a6b-4258-a6e2-5634dc8a98fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:42.735167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c7b48-8a6b-4258-a6e2-5634dc8a98fa service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:42.735499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c7b48-8a6b-4258-a6e2-5634dc8a98fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:42.735831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c7b48-8a6b-4258-a6e2-5634dc8a98fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:42.746339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c7b48-8a6b-4258-a6e2-5634dc8a98fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:42.749401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c7b48-8a6b-4258-a6e2-5634dc8a98fa service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:42.758946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 887/1781] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:43.207030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec08043-84f4-472e-8f1f-a70cad0851cc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.207563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec08043-84f4-472e-8f1f-a70cad0851cc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.207853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec08043-84f4-472e-8f1f-a70cad0851cc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.209792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 895/1782] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:43.220200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.220764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.220764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.249911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:43.256707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:43.269963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:43.433651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 888/1783] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:40:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:43.449992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec08043-84f4-472e-8f1f-a70cad0851cc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:43.456657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fec08043-84f4-472e-8f1f-a70cad0851cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:43.486544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec08043-84f4-472e-8f1f-a70cad0851cc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:43.493101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:43.493779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:43.494581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:43.495031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:43.508252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:43.510136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:43.528694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 896/1784] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:44.042428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.044715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:44.163553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc3f9b-5449-41cd-90d8-01f196483b7c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.168051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.168940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.169334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.169590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] PATH_INFO: `/v3/users/24dd7b07950541d3b0307ab268ef87a0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.170819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=24dd7b07950541d3b0307ab268ef87a0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.172752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.223799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24dd7b07950541d3b0307ab268ef87a0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.224449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24dd7b07950541d3b0307ab268ef87a0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.225046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24dd7b07950541d3b0307ab268ef87a0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.236787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24dd7b07950541d3b0307ab268ef87a0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.243940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24dd7b07950541d3b0307ab268ef87a0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.256705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 889/1785] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:44 2022] DELETE /identity/v3/users/24dd7b07950541d3b0307ab268ef87a0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:44.464533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.480570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c9d73-f67a-4dae-8820-2f939c58a77d service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.533724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:44.546978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.548115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.548575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.548815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] PATH_INFO: `/v3/projects/6e4c207993954b828bc4ffb06b456952` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.550040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4c207993954b828bc4ffb06b456952)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.552209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.611939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4c207993954b828bc4ffb06b456952', 'request_id': 'req-649def11-c3b2-41fe-bc0f-57c8c74f529a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.622578 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4c207993954b828bc4ffb06b456952', 'request_id': 'req-649def11-c3b2-41fe-bc0f-57c8c74f529a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.632290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4c207993954b828bc4ffb06b456952', 'request_id': 'req-649def11-c3b2-41fe-bc0f-57c8c74f529a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.648155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] The token cache is being invalidate because project 6e4c207993954b828bc4ffb06b456952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:44.648302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.648931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.650847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 897/1786] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:44 2022] DELETE /identity/v3/projects/6e4c207993954b828bc4ffb06b456952 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:44.655609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:44.657100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:44.673575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe8cae-a451-4ead-8290-6007b25db9eb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:44.705641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:44.802176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:44.803192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:44.803540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:44.803872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] PATH_INFO: `/v3/users/14a2b02b3bcb47658cfb7b4d966bc731` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:44.805220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a2b02b3bcb47658cfb7b4d966bc731)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:44.807404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:44.855818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a2b02b3bcb47658cfb7b4d966bc731'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.856454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a2b02b3bcb47658cfb7b4d966bc731'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.857058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a2b02b3bcb47658cfb7b4d966bc731'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.869127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a2b02b3bcb47658cfb7b4d966bc731'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.875847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-604cc023-7af6-4b50-aea1-b249208fc26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a2b02b3bcb47658cfb7b4d966bc731'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:44.890607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 890/1787] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:44 2022] DELETE /identity/v3/users/14a2b02b3bcb47658cfb7b4d966bc731 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:45.144342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.145859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:45.163033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649def11-c3b2-41fe-bc0f-57c8c74f529a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.235795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.236976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.237398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.237976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] PATH_INFO: `/v3/projects/e834bbaaea9a4355b10e06c9ececb469` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.239890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e834bbaaea9a4355b10e06c9ececb469)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.241959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.320780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e834bbaaea9a4355b10e06c9ececb469', 'request_id': 'req-cf00ca59-a571-4c62-9404-48e1b39fda24'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:45.327155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e834bbaaea9a4355b10e06c9ececb469', 'request_id': 'req-cf00ca59-a571-4c62-9404-48e1b39fda24'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:45.333431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e834bbaaea9a4355b10e06c9ececb469', 'request_id': 'req-cf00ca59-a571-4c62-9404-48e1b39fda24'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:45.348874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] The token cache is being invalidate because project e834bbaaea9a4355b10e06c9ececb469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:45.350707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:45.350707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf00ca59-a571-4c62-9404-48e1b39fda24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:45.351628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 898/1788] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:45 2022] DELETE /identity/v3/projects/e834bbaaea9a4355b10e06c9ececb469 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:45.359220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.359663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.359766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.380436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:45.381744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:45.442133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 891/1789] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:45.451745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.452107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.452390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.470994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:45.472216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:45.480387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 899/1790] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:45.484865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.494777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5073c3b-0da8-4258-878b-69fb9bb25902 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.500561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.501499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.501940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.502268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.503011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:45.503357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.505168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.518229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 892/1791] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:45.522714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.528986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:45.534254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d248fb-84c3-4ecb-aebd-36a8dcc9f9fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.541344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.542443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.543002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.543286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.544477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.547238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.577230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 900/1792] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:45.582707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.594860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676be05-efc4-4535-a989-93d57166a6fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.628673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:45.657611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.658310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.658595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.658898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.659613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.661548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.686964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 893/1793] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:45.688536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.699803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e0e20-2a29-4e07-a3b6-2d34ce3e5c3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.707180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.708039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.708350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.708662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.709582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.712673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.723866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 901/1794] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:40:45.727901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.738102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669c876-29e6-4709-8569-998cb79c9df9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.743122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99b2ed-aca5-4624-9182-e43b930b866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.743788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b2ed-aca5-4624-9182-e43b930b866d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.744183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b2ed-aca5-4624-9182-e43b930b866d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.744460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99b2ed-aca5-4624-9182-e43b930b866d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.745189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99b2ed-aca5-4624-9182-e43b930b866d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.746819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99b2ed-aca5-4624-9182-e43b930b866d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.756866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 894/1795] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:40:45.761043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:45.771586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74486a09-e8d9-438c-b771-cd3ecd3e2d3c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:45.778199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:45.779153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.779430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.779701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] PATH_INFO: `/v3/projects/6dba32326fdc4f23a19300bc489f80f0/users/354aade93d534182808d77db3fd4fe9c/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.795027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dba32326fdc4f23a19300bc489f80f0, user_id=354aade93d534182808d77db3fd4fe9c, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:45.797756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:45.814341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 902/1796] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:45 2022] PUT /identity/v3/projects/6dba32326fdc4f23a19300bc489f80f0/users/354aade93d534182808d77db3fd4fe9c/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:45.824856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:45.825148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:45.825527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:45.853821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] MFA Rules not processed for user `354aade93d534182808d77db3fd4fe9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:45.855131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:45.930697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 895/1797] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:45.935926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.001237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734f399-dfd1-49d0-b3b9-bf97a430404d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.006210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.006912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.007188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.007462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.008153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.010023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.037661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 903/1798] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:45 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:46.041834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.053169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69372eb-3b26-41b1-86da-e028285a11a7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.118743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.119716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.120039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.120324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.121110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.123606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.145839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 896/1799] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:46 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:46.149922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.161548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee08cd-9486-4f4a-a665-9689c34f8061 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.166890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.167555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.167955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.168231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.169206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.170621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.182037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 904/1800] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:46.186784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.197452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bf393-f0a1-4654-8b60-200222c692da admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.202742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.203700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.203977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.204246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] PATH_INFO: `/v3/projects/b666f2abaa71466f8d66a72ca219ee27/users/ab66ec9948f14406a467a068b579d670/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.221185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b666f2abaa71466f8d66a72ca219ee27, user_id=ab66ec9948f14406a467a068b579d670, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.224101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.239089 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 897/1801] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:46 2022] PUT /identity/v3/projects/b666f2abaa71466f8d66a72ca219ee27/users/ab66ec9948f14406a467a068b579d670/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:46.243066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.253277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783b77-28e6-49e2-92fd-3c36e085cf95 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.319001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedc05c4-0c61-4ebb-893e-bf258324c8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.319690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc05c4-0c61-4ebb-893e-bf258324c8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.319973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc05c4-0c61-4ebb-893e-bf258324c8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.320223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedc05c4-0c61-4ebb-893e-bf258324c8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.321101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc05c4-0c61-4ebb-893e-bf258324c8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.322768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedc05c4-0c61-4ebb-893e-bf258324c8b5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.339055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 905/1802] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:46.340498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.355481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bc1dc-df66-4c00-bc26-813b85fa1ee8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.362188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.363468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.363813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.364151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] PATH_INFO: `/v3/projects/b666f2abaa71466f8d66a72ca219ee27/users/ab66ec9948f14406a467a068b579d670/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.366801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b666f2abaa71466f8d66a72ca219ee27, user_id=ab66ec9948f14406a467a068b579d670, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.369280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.371881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:46.386845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 898/1803] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:46 2022] PUT /identity/v3/projects/b666f2abaa71466f8d66a72ca219ee27/users/ab66ec9948f14406a467a068b579d670/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:46.396601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.396943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.397356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.425900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] MFA Rules not processed for user `ab66ec9948f14406a467a068b579d670`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:46.427130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:46.431108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:46.494780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 906/1804] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:46.506686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.508685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:46.568621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcbc7d-ce86-4070-a11e-4dc84d8fa286 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.572624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.573265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.573541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.573810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.583363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.584801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.594583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 899/1805] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:46.678375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:46.679783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:46.692126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8d416-c55a-4172-9cbc-41c696b0132b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:46.696163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193f716-857b-4186-baac-5d7370c3c80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:46.696810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193f716-857b-4186-baac-5d7370c3c80c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:46.697087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193f716-857b-4186-baac-5d7370c3c80c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:46.697358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193f716-857b-4186-baac-5d7370c3c80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:46.767953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193f716-857b-4186-baac-5d7370c3c80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:46.769393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193f716-857b-4186-baac-5d7370c3c80c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:46.780864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 907/1806] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:49.654043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:49.779370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1b364-e3b9-46aa-91fa-e1fab75af025 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:49.792283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:49.798868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:49.799226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:49.799677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] PATH_INFO: `/v3/users/354aade93d534182808d77db3fd4fe9c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:49.801525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=354aade93d534182808d77db3fd4fe9c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:49.803680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:49.951543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354aade93d534182808d77db3fd4fe9c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:49.952712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354aade93d534182808d77db3fd4fe9c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:49.960124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354aade93d534182808d77db3fd4fe9c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:49.996725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354aade93d534182808d77db3fd4fe9c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:50.011662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354aade93d534182808d77db3fd4fe9c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:50.043711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 900/1807] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:49 2022] DELETE /identity/v3/users/354aade93d534182808d77db3fd4fe9c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:50.051956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbb274-a13c-46f7-9634-3b877abaf133 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.051956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbb274-a13c-46f7-9634-3b877abaf133 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.051956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbb274-a13c-46f7-9634-3b877abaf133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.088369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bcfbb274-a13c-46f7-9634-3b877abaf133 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:50.090347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bcfbb274-a13c-46f7-9634-3b877abaf133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:50.246482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 908/1808] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:50.401305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:50.403834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:50.438661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:50.532344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f050e-703c-49fc-896d-5f4b8eec1c2d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:50.543503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df814-2cad-4bd6-8e5c-28159a2534c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:50.544486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df814-2cad-4bd6-8e5c-28159a2534c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:50.544844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df814-2cad-4bd6-8e5c-28159a2534c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:50.547704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df814-2cad-4bd6-8e5c-28159a2534c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:50.574884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df814-2cad-4bd6-8e5c-28159a2534c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:50.577399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df814-2cad-4bd6-8e5c-28159a2534c4 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:50.611582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 901/1809] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:51.160581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cc1c3-45bf-4f0f-a451-59938d9e84fd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:51.162502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cc1c3-45bf-4f0f-a451-59938d9e84fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:51.163310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cc1c3-45bf-4f0f-a451-59938d9e84fd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:51.165855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 909/1810] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:40:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:40:51.178220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:51.178839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:51.179444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:51.239283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:51.243640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:51.374922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 902/1811] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:40:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:51.484077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cc1c3-45bf-4f0f-a451-59938d9e84fd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:51.487947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6e2cc1c3-45bf-4f0f-a451-59938d9e84fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:51.528889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cc1c3-45bf-4f0f-a451-59938d9e84fd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:51.537868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:51.538998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:51.539455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:51.540019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:51.553125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:51.555482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:51.571025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 910/1812] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:51.882801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:51.884303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:51.923628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec03f0e-88a7-4294-9da7-4f012c33d037 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:51.945356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:51.946486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:51.946814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:51.947102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] PATH_INFO: `/v3/projects/6dba32326fdc4f23a19300bc489f80f0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:51.951808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dba32326fdc4f23a19300bc489f80f0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:51.959507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:52.080719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dba32326fdc4f23a19300bc489f80f0', 'request_id': 'req-7f715787-964f-4259-83cf-d61302b46b8a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.089324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dba32326fdc4f23a19300bc489f80f0', 'request_id': 'req-7f715787-964f-4259-83cf-d61302b46b8a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.099827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dba32326fdc4f23a19300bc489f80f0', 'request_id': 'req-7f715787-964f-4259-83cf-d61302b46b8a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.121134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] The token cache is being invalidate because project 6dba32326fdc4f23a19300bc489f80f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:52.121912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.123593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.125349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 903/1813] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:51 2022] DELETE /identity/v3/projects/6dba32326fdc4f23a19300bc489f80f0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:52.130677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:52.167009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a9715-426b-471f-8fe2-fee9b64a3ed4 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:52.231424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:52.320281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:52.322260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:52.323200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:52.323621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] PATH_INFO: `/v3/users/ab66ec9948f14406a467a068b579d670` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:52.325931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab66ec9948f14406a467a068b579d670)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:52.328573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:52.413563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab66ec9948f14406a467a068b579d670'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.414345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab66ec9948f14406a467a068b579d670'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.428627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab66ec9948f14406a467a068b579d670'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.429973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab66ec9948f14406a467a068b579d670'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.446188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab66ec9948f14406a467a068b579d670'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:52.464098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 911/1814] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:52 2022] DELETE /identity/v3/users/ab66ec9948f14406a467a068b579d670 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:53.044259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.046488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.061326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715787-964f-4259-83cf-d61302b46b8a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.098190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.137765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.138687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.139021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.139350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] PATH_INFO: `/v3/users/be848e24e08f493ab165760a30d3d8b7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.141971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] RBAC: Authorizing `identity:delete_user(user_id=be848e24e08f493ab165760a30d3d8b7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.144004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.216378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.218654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.234563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be848e24e08f493ab165760a30d3d8b7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.235718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be848e24e08f493ab165760a30d3d8b7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.237029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be848e24e08f493ab165760a30d3d8b7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.248477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ba3de-25b3-414f-8dae-ae1c98bede23 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.254709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.255187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be848e24e08f493ab165760a30d3d8b7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.255637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.255920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.256192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] PATH_INFO: `/v3/projects/b666f2abaa71466f8d66a72ca219ee27` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.257382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b666f2abaa71466f8d66a72ca219ee27)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.261971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.264201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-150a809e-e521-4ba6-bdbc-57fecb07debd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be848e24e08f493ab165760a30d3d8b7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.291197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 904/1815] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:40:53 2022] DELETE /identity/v3/users/be848e24e08f493ab165760a30d3d8b7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:53.297682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.297979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.298255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.326426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:53.327943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.335206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b666f2abaa71466f8d66a72ca219ee27', 'request_id': 'req-5a9f989f-ba10-4568-9ad2-df9d2a418469'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.343382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b666f2abaa71466f8d66a72ca219ee27', 'request_id': 'req-5a9f989f-ba10-4568-9ad2-df9d2a418469'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.351584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b666f2abaa71466f8d66a72ca219ee27', 'request_id': 'req-5a9f989f-ba10-4568-9ad2-df9d2a418469'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.379808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] The token cache is being invalidate because project b666f2abaa71466f8d66a72ca219ee27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:53.380103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.381699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5a9f989f-ba10-4568-9ad2-df9d2a418469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:53.384659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 912/1816] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:53 2022] DELETE /identity/v3/projects/b666f2abaa71466f8d66a72ca219ee27 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:53.392759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a27c-8b79-462c-903c-3fd4c785fe36 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.393045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a27c-8b79-462c-903c-3fd4c785fe36 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.393327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a27c-8b79-462c-903c-3fd4c785fe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.417750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9884a27c-8b79-462c-903c-3fd4c785fe36 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:53.418995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9884a27c-8b79-462c-903c-3fd4c785fe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.436800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 905/1817] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:53.450433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.452262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.454677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.533169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151aac8-ea29-4b0b-b9c4-f441437643f8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.542612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 913/1818] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:53.544986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.545374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.546318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.548848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.549627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.550146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.550251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.558701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:53.561550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.563857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.572167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:53.573406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:53.575507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 906/1819] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:53.582564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 914/1820] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:53.587157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.598904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840145-160d-4a12-b9b2-c0def061f7c5 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.604852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.606703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.606703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.607088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.607907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:53.608306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.610233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.625670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 907/1821] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:40:53.630181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.642640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081563d8-d72b-4c15-8706-1ba2130ad3ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.655113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.655857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.656142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.656416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.658505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.664178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.716962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 915/1822] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:53.721095 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.739625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9cb3-ad5f-4be3-a4ba-c0910cf7701a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.857750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.858504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.858877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.861024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.861419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.865720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:53.915088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 908/1823] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:53 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:53.920700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:53.944934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c493f6a-3df9-476a-95a8-4a52142f49de admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:53.965918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:53.967912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:53.968369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:53.969446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:53.970346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:53.975088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.004611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 916/1824] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:54.009591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.029850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dedb5-982b-4be3-b4a7-17acd6eb8f3a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.043344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.045009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.045009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.045650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.045853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.051186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.066272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 909/1825] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:54.070445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.075359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.077569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:54.083163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7469085-a57d-4415-ba53-3182dbce8cdf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.088449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e484878-6f71-4421-af81-090b54dd97ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.089616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484878-6f71-4421-af81-090b54dd97ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.089955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484878-6f71-4421-af81-090b54dd97ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.090401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e484878-6f71-4421-af81-090b54dd97ae admin admin] PATH_INFO: `/v3/projects/0c9c3648f2b741d18eb0e56214032f24/users/2f7a3a959cb44b67a9374b155e1ad946/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.096196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de9bd4-c80e-42bc-b53d-0a01a7dbb54d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.103987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.104885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.105164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.105597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] PATH_INFO: `/v3/projects/069dbbe5f20f48b480d1b8f5e93abb1f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.106992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069dbbe5f20f48b480d1b8f5e93abb1f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.109670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.122745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e484878-6f71-4421-af81-090b54dd97ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9c3648f2b741d18eb0e56214032f24, user_id=2f7a3a959cb44b67a9374b155e1ad946, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.128625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e484878-6f71-4421-af81-090b54dd97ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.152988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 917/1826] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:54 2022] PUT /identity/v3/projects/0c9c3648f2b741d18eb0e56214032f24/users/2f7a3a959cb44b67a9374b155e1ad946/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:54.172605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.175612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.176712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.214542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069dbbe5f20f48b480d1b8f5e93abb1f', 'request_id': 'req-481031e5-d384-47b3-8ac6-bc9e139e0e23'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:54.225727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] MFA Rules not processed for user `2f7a3a959cb44b67a9374b155e1ad946`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:54.227809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069dbbe5f20f48b480d1b8f5e93abb1f', 'request_id': 'req-481031e5-d384-47b3-8ac6-bc9e139e0e23'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:54.229103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:54.241877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069dbbe5f20f48b480d1b8f5e93abb1f', 'request_id': 'req-481031e5-d384-47b3-8ac6-bc9e139e0e23'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:54.267852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] The token cache is being invalidate because project 069dbbe5f20f48b480d1b8f5e93abb1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:40:54.268145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:54.268938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-481031e5-d384-47b3-8ac6-bc9e139e0e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:40:54.272094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 910/1827] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:40:54 2022] DELETE /identity/v3/projects/069dbbe5f20f48b480d1b8f5e93abb1f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:54.277563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9abf58-9b93-4e00-b530-c1f51a75648d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.277848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9abf58-9b93-4e00-b530-c1f51a75648d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.278127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9abf58-9b93-4e00-b530-c1f51a75648d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.320943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0f9abf58-9b93-4e00-b530-c1f51a75648d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:54.323241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0f9abf58-9b93-4e00-b530-c1f51a75648d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:54.363280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 918/1828] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:54.367679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.371050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:54.408557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:54.438060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 911/1829] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:54.446200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.447167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.447571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.473985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:54.475847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:54.479438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:54.486991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbbce8-a910-45f3-a8d8-d63fc9be625b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.496597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.496597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.496597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.496597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.497114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.498097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 912/1830] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:40:54.498537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.499468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.513973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf4a81-5f9b-4d1f-ad49-99ef4738c5c0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.523875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.525165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.525512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.525870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.526593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:40:54.527336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.529243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.542054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 919/1831] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/projects => generated 369 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:54.558951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 913/1832] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:40:54.559166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.560152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.586345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9b49-fd2f-4a40-b1cb-2b43890d47be admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.660209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f7d93-ea3c-4bd9-a390-b26714c2ad4e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.665830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.666869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.667118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.667388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.668200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.670239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.675643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.676448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.676826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.677336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.677993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.680031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.699353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 920/1833] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:54.700384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.712086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4c2eb-c480-44eb-9da6-fe69c7f27a40 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.717471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.718193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.718590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.718903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.719697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.721498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.732795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 914/1834] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:54.734399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.744346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 921/1835] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:54.745492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.761859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ec087-ec65-4ddb-a5ee-3c37ef5540cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.821077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c1d3-19ab-4249-ae24-5709b43480b0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.826789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.827478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.827762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.828033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.828865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.831158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.846121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.847371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.847844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.848293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] PATH_INFO: `/v3/projects/9f554f3f0c6845e8bc824c9db2d2ee33/users/f3a85195a0244d31b69d10813a526891/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.863015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 915/1836] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:54 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:54.868387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.869786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f554f3f0c6845e8bc824c9db2d2ee33, user_id=f3a85195a0244d31b69d10813a526891, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.873724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.881226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fed0a8-a6c3-442e-811b-9e26c1f1c1a5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.887963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:54.889121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:54.889449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:54.889965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:54.891015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:54.892830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:54.893406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 922/1837] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:54 2022] PUT /identity/v3/projects/9f554f3f0c6845e8bc824c9db2d2ee33/users/f3a85195a0244d31b69d10813a526891/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:40:54.897401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.919011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 916/1838] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:54.921274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:54.941670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384e826-72ad-4dc8-afa6-b48003b0fe0a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:54.976491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:54.996667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:55.012076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c3308-424f-4a29-9d20-8d0dc3b14cb5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.025099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.026484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.027848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.028378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.029398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.031254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.036403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.037258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.037756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.038106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.038991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.040723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.051392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 923/1839] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:55.054814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 917/1840] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:55.056229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.057557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.069904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d4aae-23ea-4fa6-be03-e1c9f138cbfa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.076016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a97ba-839d-4438-bd49-ae718a170c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.077147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a97ba-839d-4438-bd49-ae718a170c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.077520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a97ba-839d-4438-bd49-ae718a170c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.077969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a97ba-839d-4438-bd49-ae718a170c6b admin admin] PATH_INFO: `/v3/projects/9f554f3f0c6845e8bc824c9db2d2ee33/users/f3a85195a0244d31b69d10813a526891/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.080967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a97ba-839d-4438-bd49-ae718a170c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f554f3f0c6845e8bc824c9db2d2ee33, user_id=f3a85195a0244d31b69d10813a526891, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.082588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f0194-562b-4b80-958f-6b5b6434ab51 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.084674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a97ba-839d-4438-bd49-ae718a170c6b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.099357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d177a-47db-4bc1-b9e8-97ae39e36d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.100380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d177a-47db-4bc1-b9e8-97ae39e36d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.100616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d177a-47db-4bc1-b9e8-97ae39e36d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.100943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d177a-47db-4bc1-b9e8-97ae39e36d4b admin admin] PATH_INFO: `/v3/projects/4877ecc00a454c4db997977468cb7e30/users/b2b28dcf77bf4325a5a6d983c579f68c/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.102846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 924/1841] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:55 2022] PUT /identity/v3/projects/9f554f3f0c6845e8bc824c9db2d2ee33/users/f3a85195a0244d31b69d10813a526891/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:55.119835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.120104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.124201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.146809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d177a-47db-4bc1-b9e8-97ae39e36d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4877ecc00a454c4db997977468cb7e30, user_id=b2b28dcf77bf4325a5a6d983c579f68c, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.169329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d177a-47db-4bc1-b9e8-97ae39e36d4b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.199858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 918/1842] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:55 2022] PUT /identity/v3/projects/4877ecc00a454c4db997977468cb7e30/users/b2b28dcf77bf4325a5a6d983c579f68c/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:55.212342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.212765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.213279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.217527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] MFA Rules not processed for user `f3a85195a0244d31b69d10813a526891`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:55.218923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:55.276148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] MFA Rules not processed for user `b2b28dcf77bf4325a5a6d983c579f68c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:55.280033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:55.416010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 919/1843] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:55.426145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.431131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 925/1844] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:55.456281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.460566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:55.539031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976d770-c9fa-4fae-97c0-e84c25641b71 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.546144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.547680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.547680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.548002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.548802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.550808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.613171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 920/1845] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:55 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:55.614431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.630544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0b262-571b-439e-a27b-841c2508ef7b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.683840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:55.853548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.853814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.854097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.854366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.868039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ef052-cee1-4567-9f16-f3b35bc8781e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.874908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.875867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.876251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.876624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.877727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.882344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.916999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:55.933482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 921/1846] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:55 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:55.936015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:55.951958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef133c-0c67-4074-b311-399f7e7faa83 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:55.962735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:55.965319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:55.965319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:55.965319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:55.965319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:55.966648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:55.988005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 922/1847] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:55.989593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.007037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b90d-7685-48f5-8b44-797e61696e27 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.019839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.020196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.020541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.020822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.021091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.021726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.021846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.034341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.044228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 926/1848] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:56.055640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 923/1849] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:56.059703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.075146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0157d-1904-42ba-867c-cc54e6ddca94 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.083104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f412048-a73d-48df-acfc-ff9e50b632fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.084118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f412048-a73d-48df-acfc-ff9e50b632fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.084542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f412048-a73d-48df-acfc-ff9e50b632fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.084825 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f412048-a73d-48df-acfc-ff9e50b632fa admin admin] PATH_INFO: `/v3/projects/b1a56f14b09e44429b9126694fed554a/users/41c19550924e4bae84544fd41652b93c/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.102347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f412048-a73d-48df-acfc-ff9e50b632fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a56f14b09e44429b9126694fed554a, user_id=41c19550924e4bae84544fd41652b93c, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.105205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f412048-a73d-48df-acfc-ff9e50b632fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.120425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.122552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:56.131792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 927/1850] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:56 2022] PUT /identity/v3/projects/b1a56f14b09e44429b9126694fed554a/users/41c19550924e4bae84544fd41652b93c/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:56.140841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21eb1e-90d3-4547-a9c4-9193d533b1c5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.150161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.150454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.150729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.187496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] MFA Rules not processed for user `41c19550924e4bae84544fd41652b93c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:56.188741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:56.199132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.199796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.200215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.200488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.264848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.266369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.271370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 928/1851] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:56.273193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.280406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 924/1852] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:40:56.349562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4060b61-63d7-4690-a88f-cf90f0496a37 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.355372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.356388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.356801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.357143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.357973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.360306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.425343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 929/1853] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:40:56 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:40:56.426572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.444827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf6c0b-8509-4115-9e2c-db29b9df6677 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.547376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:56.595559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.596311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.596666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.596797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.597501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.599925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.646565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 925/1854] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:40:56 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:40:56.647856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.660292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4647d-cb85-4968-9aef-dc26f0cbceb7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.666188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.676823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.676823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.678375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.679370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.681171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.683932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:40:56.701918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.714055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b845a-a958-43f0-98ef-8402f55c7499 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.716352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 930/1855] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:56.728476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.729396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.729677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.729950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] PATH_INFO: `/v3/projects/7969ad6e7ad14636989adc92d768be3f/users/262e96651a604ed7afbf98219e8d9458/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.768940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969ad6e7ad14636989adc92d768be3f, user_id=262e96651a604ed7afbf98219e8d9458, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.771673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.813988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 926/1856] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:56 2022] PUT /identity/v3/projects/7969ad6e7ad14636989adc92d768be3f/users/262e96651a604ed7afbf98219e8d9458/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:56.818338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.830429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085fc73-351d-44d4-a466-d1de44fb698b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.914432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe2c35-d5e0-4e63-b3bf-3872ac086464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.915446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe2c35-d5e0-4e63-b3bf-3872ac086464 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.915997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe2c35-d5e0-4e63-b3bf-3872ac086464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.915997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe2c35-d5e0-4e63-b3bf-3872ac086464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.916792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe2c35-d5e0-4e63-b3bf-3872ac086464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.918815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe2c35-d5e0-4e63-b3bf-3872ac086464 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:56.930247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 931/1857] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:40:56.934469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:56.949781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0454fc-0983-49fb-9219-4c4669608aeb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:56.962702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:56.965282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:56.965908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:56.966346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] PATH_INFO: `/v3/projects/7969ad6e7ad14636989adc92d768be3f/users/262e96651a604ed7afbf98219e8d9458/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:56.969270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969ad6e7ad14636989adc92d768be3f, user_id=262e96651a604ed7afbf98219e8d9458, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:56.972378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:57.003511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 927/1858] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:40:56 2022] PUT /identity/v3/projects/7969ad6e7ad14636989adc92d768be3f/users/262e96651a604ed7afbf98219e8d9458/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:40:57.019745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.020072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.020334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.076103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] MFA Rules not processed for user `262e96651a604ed7afbf98219e8d9458`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:40:57.077342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:40:57.157643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 932/1859] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:40:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:40:57.170023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:57.249038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd8e22-8223-43ed-85e4-dda256b80b0c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:57.253886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:57.254547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:57.254859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:57.255130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:57.315512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:57.316958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:57.326254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 928/1860] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:40:58.882612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:40:58.897407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cec4e-f1fb-4e91-989e-2889b962a130 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:40:58.904379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:40:58.905030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:40:58.905306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:40:58.905561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:40:58.965388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:40:58.966979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:40:58.977067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 933/1861] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:40:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:00.128708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:00.138138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659f632-a69d-4bf8-bb66-5da41cdc3fa6 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:00.141955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79069f-597c-4930-9e7b-8761f0f2bb94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:00.142601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79069f-597c-4930-9e7b-8761f0f2bb94 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.142926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79069f-597c-4930-9e7b-8761f0f2bb94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.143198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79069f-597c-4930-9e7b-8761f0f2bb94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.152874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79069f-597c-4930-9e7b-8761f0f2bb94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:00.154354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79069f-597c-4930-9e7b-8761f0f2bb94 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:00.164122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 929/1862] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:00.394572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:00.412954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:00.461194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924bd5-4f5f-4a06-a50b-ad7ce064f719 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:00.466033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:00.466977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.467255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.467669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] PATH_INFO: `/v3/users/2f7a3a959cb44b67a9374b155e1ad946` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.468862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7a3a959cb44b67a9374b155e1ad946)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:00.470821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:00.537715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7a3a959cb44b67a9374b155e1ad946'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:00.538478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7a3a959cb44b67a9374b155e1ad946'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:00.555435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7a3a959cb44b67a9374b155e1ad946'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:00.556285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7a3a959cb44b67a9374b155e1ad946'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:00.570552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7a3a959cb44b67a9374b155e1ad946'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:00.590782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 934/1863] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:00 2022] DELETE /identity/v3/users/2f7a3a959cb44b67a9374b155e1ad946 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:00.595422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.595728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.596223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.615624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:00.616844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:00.652846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:00.691926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 930/1864] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:00.701648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:00.703176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:00.760764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f669df0-b44f-4425-8071-0108cd717503 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:00.764507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:00.765166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:00.765430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:00.765695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:00.774813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:00.776282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:00.786643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 935/1865] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:01.171693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.173988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:01.189623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4cf62-4a06-48db-9a94-83ef29b866a7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:01.195440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:01.196628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.197617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.198127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] PATH_INFO: `/v3/projects/0c9c3648f2b741d18eb0e56214032f24` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.199872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9c3648f2b741d18eb0e56214032f24)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:01.202676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:01.270117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9c3648f2b741d18eb0e56214032f24', 'request_id': 'req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.277234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9c3648f2b741d18eb0e56214032f24', 'request_id': 'req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.284502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9c3648f2b741d18eb0e56214032f24', 'request_id': 'req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.297709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] The token cache is being invalidate because project 0c9c3648f2b741d18eb0e56214032f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:01.298007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.298626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.300297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 931/1866] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:01 2022] DELETE /identity/v3/projects/0c9c3648f2b741d18eb0e56214032f24 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:01.304390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.315185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5609c-24dd-4034-a451-3aa960ae22cc service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:01.366161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:01.394090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:01.395133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.395681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.395681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] PATH_INFO: `/v3/users/f3a85195a0244d31b69d10813a526891` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.397261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a85195a0244d31b69d10813a526891)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:01.399575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:01.451078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a85195a0244d31b69d10813a526891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.451716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a85195a0244d31b69d10813a526891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.458641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a85195a0244d31b69d10813a526891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.459305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a85195a0244d31b69d10813a526891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.473977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-98982b87-b599-4442-b231-b5b6b95cb2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a85195a0244d31b69d10813a526891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:01.491635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 936/1867] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:01 2022] DELETE /identity/v3/users/f3a85195a0244d31b69d10813a526891 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:01.849493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:01.851094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:01.874541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5bbb0-e460-4ee1-aaec-9eef7725f8bc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:01.946624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:01.947546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:01.947826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:01.948102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] PATH_INFO: `/v3/projects/9f554f3f0c6845e8bc824c9db2d2ee33` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:01.949582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f554f3f0c6845e8bc824c9db2d2ee33)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:01.951363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.004459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f554f3f0c6845e8bc824c9db2d2ee33', 'request_id': 'req-30ddd1ca-72ee-4834-a549-b601b53fa516'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:02.014090 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f554f3f0c6845e8bc824c9db2d2ee33', 'request_id': 'req-30ddd1ca-72ee-4834-a549-b601b53fa516'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:02.019398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:02.025933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f554f3f0c6845e8bc824c9db2d2ee33', 'request_id': 'req-30ddd1ca-72ee-4834-a549-b601b53fa516'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:02.050534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] The token cache is being invalidate because project 9f554f3f0c6845e8bc824c9db2d2ee33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:02.051206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:02.052363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30ddd1ca-72ee-4834-a549-b601b53fa516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:02.055317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 932/1868] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:01 2022] DELETE /identity/v3/projects/9f554f3f0c6845e8bc824c9db2d2ee33 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:02.060788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.061144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.061491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.083323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:02.084751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:02.181794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 937/1869] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:02.183544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.183983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.184272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.204372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:02.205611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:02.213893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 933/1870] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:02.218323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.229621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce354f-bfcf-4b9a-b577-75ab672a87d5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.234962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.236094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.236592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.237084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.238619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:02.239203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.242396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.269981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 938/1871] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:02.270600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.282288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c25421-452c-4074-b3d7-7b3ea14df68a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.287693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.288527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.288876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.289249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.290100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.292324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.321036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 934/1872] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:02.325573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.337517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d24e46-1b62-4337-8111-5cbb4b5f1771 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.382488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:02.400004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.401148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.401457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.402070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.402933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.404861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.431978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 939/1873] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:02.437291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.448016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d247888-98d5-4c13-a6cf-0ea263b958d1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.454537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.455254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.455544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.455806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.456535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.458157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.473546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 935/1874] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:02.476666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.492932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421fcff-f206-46cc-9762-05b4f433f4f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.501070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae98844-5560-4c12-81ac-e93844045c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.501760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae98844-5560-4c12-81ac-e93844045c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.502037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae98844-5560-4c12-81ac-e93844045c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.502316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae98844-5560-4c12-81ac-e93844045c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.503082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae98844-5560-4c12-81ac-e93844045c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.504691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae98844-5560-4c12-81ac-e93844045c7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.517279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 940/1875] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:02.521653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.535347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c27920-257f-483a-98d2-36cd41c9b2e4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.540735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.541637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.541919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.542192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] PATH_INFO: `/v3/projects/ffc7ad629dba47419097b2046a6fe635/users/d4c159b627e147f8aa3dadf2e91989c9/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.557426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc7ad629dba47419097b2046a6fe635, user_id=d4c159b627e147f8aa3dadf2e91989c9, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.560349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.576049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 936/1876] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:02 2022] PUT /identity/v3/projects/ffc7ad629dba47419097b2046a6fe635/users/d4c159b627e147f8aa3dadf2e91989c9/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:02.588372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.588671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.588948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.619564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] MFA Rules not processed for user `d4c159b627e147f8aa3dadf2e91989c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:02.621178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:02.747116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 941/1877] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:02.752238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.914884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5cd68-eac7-4c9f-8eb4-4680b740f507 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:02.922037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:02.922966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:02.923304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:02.923581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:02.924287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:02.926453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:02.968089 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 937/1878] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/projects => generated 387 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:02.972324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:02.986806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caf5f6-f394-4c80-af9d-9e4e402c62ec None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.093884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.094708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.095030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.095297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.096128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.098215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.137061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 942/1879] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:02 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:03.140245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.154610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2df116-c126-42c8-909f-585ca16b8c04 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.161641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.162368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.162733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.163040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.163780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.165580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.183285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 938/1880] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:03.187831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.201132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99546bf-5d8b-4590-aa56-48a33047a1e5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.206700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.207915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.208270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.208602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] PATH_INFO: `/v3/projects/dc33b0bff0604bf5bdb9da47edfc1f92/users/278c943a50f2423fa6aa77ea33a9e0c8/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.227974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc33b0bff0604bf5bdb9da47edfc1f92, user_id=278c943a50f2423fa6aa77ea33a9e0c8, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.230870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.254796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 943/1881] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:03 2022] PUT /identity/v3/projects/dc33b0bff0604bf5bdb9da47edfc1f92/users/278c943a50f2423fa6aa77ea33a9e0c8/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:03.256273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.268635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e62f6d-de07-48ee-ae4d-e73a312b2e56 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.337102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:03.345972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd08a8-a408-4840-ab35-26e283e2e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.346887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd08a8-a408-4840-ab35-26e283e2e2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.347365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd08a8-a408-4840-ab35-26e283e2e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.347647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd08a8-a408-4840-ab35-26e283e2e2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.348639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd08a8-a408-4840-ab35-26e283e2e2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.350371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd08a8-a408-4840-ab35-26e283e2e2d9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.365494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 939/1882] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:41:03.366475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.377996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b3011-0966-40f2-bc1c-a50942f368a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.382504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.383428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.383710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.383980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] PATH_INFO: `/v3/projects/dc33b0bff0604bf5bdb9da47edfc1f92/users/278c943a50f2423fa6aa77ea33a9e0c8/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.385939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc33b0bff0604bf5bdb9da47edfc1f92, user_id=278c943a50f2423fa6aa77ea33a9e0c8, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.388650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.398981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 944/1883] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:03 2022] PUT /identity/v3/projects/dc33b0bff0604bf5bdb9da47edfc1f92/users/278c943a50f2423fa6aa77ea33a9e0c8/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:03.410995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.411280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.411555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.439969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] MFA Rules not processed for user `278c943a50f2423fa6aa77ea33a9e0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:03.441475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:03.513315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 940/1884] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:03.524181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:03.525926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:03.533053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:03.588418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505bfac-6f50-42fd-be3b-c49d5aeedda2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:03.593092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b429fd-b041-4fd7-9e3d-efeea59cf8de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:03.593741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b429fd-b041-4fd7-9e3d-efeea59cf8de service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:03.594016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b429fd-b041-4fd7-9e3d-efeea59cf8de service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:03.594285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b429fd-b041-4fd7-9e3d-efeea59cf8de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:03.607084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b429fd-b041-4fd7-9e3d-efeea59cf8de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:03.608518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b429fd-b041-4fd7-9e3d-efeea59cf8de service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:03.618088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 945/1885] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:04.244057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:04.333329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b367f1-59fc-4cff-9775-1a1c3a536db6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:04.338101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:04.339038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:04.339355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:04.339593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] PATH_INFO: `/v3/users/d4c159b627e147f8aa3dadf2e91989c9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:04.341996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c159b627e147f8aa3dadf2e91989c9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:04.346891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:04.419548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c159b627e147f8aa3dadf2e91989c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:04.420663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c159b627e147f8aa3dadf2e91989c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:04.422021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c159b627e147f8aa3dadf2e91989c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:04.444910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c159b627e147f8aa3dadf2e91989c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:04.456042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c159b627e147f8aa3dadf2e91989c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:04.460891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:04.487355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 941/1886] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:04 2022] DELETE /identity/v3/users/d4c159b627e147f8aa3dadf2e91989c9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:04.493981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:04.494270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:04.494549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:04.515712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:04.516989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:04.585903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 946/1887] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:41:04.598915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:04.600943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:04.662625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77238e-aba0-4686-8524-511176f7df02 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:04.666468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:04.667159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:04.667441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:04.667718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:04.678063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:04.679557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:04.694050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 942/1888] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:04.985448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:04.987088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:05.001356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138bff4-9ffe-47c0-949a-d1c7deba2d71 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:05.005858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:05.007820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.008089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.008356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] PATH_INFO: `/v3/projects/ffc7ad629dba47419097b2046a6fe635` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.010447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc7ad629dba47419097b2046a6fe635)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:05.012941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:05.061371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc7ad629dba47419097b2046a6fe635', 'request_id': 'req-d0722b0a-6998-4b41-9134-b6049c8e7f63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.069418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc7ad629dba47419097b2046a6fe635', 'request_id': 'req-d0722b0a-6998-4b41-9134-b6049c8e7f63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.076251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc7ad629dba47419097b2046a6fe635', 'request_id': 'req-d0722b0a-6998-4b41-9134-b6049c8e7f63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.090103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] The token cache is being invalidate because project ffc7ad629dba47419097b2046a6fe635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:05.090380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.091210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.093295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 947/1889] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:04 2022] DELETE /identity/v3/projects/ffc7ad629dba47419097b2046a6fe635 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:05.097558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:05.107534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65662012-0507-41d2-9f92-4aaef720d884 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:05.165980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:05.166969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.167256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.167521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] PATH_INFO: `/v3/users/278c943a50f2423fa6aa77ea33a9e0c8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.168892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=278c943a50f2423fa6aa77ea33a9e0c8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:05.170856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:05.223355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278c943a50f2423fa6aa77ea33a9e0c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.224048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278c943a50f2423fa6aa77ea33a9e0c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.224694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278c943a50f2423fa6aa77ea33a9e0c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.237511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278c943a50f2423fa6aa77ea33a9e0c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.244379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-62d0f94f-018f-4b86-af8e-3c4bec00cc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278c943a50f2423fa6aa77ea33a9e0c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.257942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 943/1890] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:05 2022] DELETE /identity/v3/users/278c943a50f2423fa6aa77ea33a9e0c8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:05.569902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:05.571566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:05.586854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0722b0a-6998-4b41-9134-b6049c8e7f63 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:05.624725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:05.655750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:05.656609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.656904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.657174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] PATH_INFO: `/v3/projects/dc33b0bff0604bf5bdb9da47edfc1f92` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.658413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc33b0bff0604bf5bdb9da47edfc1f92)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:05.660451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:05.719842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc33b0bff0604bf5bdb9da47edfc1f92', 'request_id': 'req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.730205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc33b0bff0604bf5bdb9da47edfc1f92', 'request_id': 'req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.739601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc33b0bff0604bf5bdb9da47edfc1f92', 'request_id': 'req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.755345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] The token cache is being invalidate because project dc33b0bff0604bf5bdb9da47edfc1f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:05.755634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.756281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4013078e-efa3-40b7-8fe4-a3b0c0f1b7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:05.758017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 948/1891] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:05 2022] DELETE /identity/v3/projects/dc33b0bff0604bf5bdb9da47edfc1f92 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:05.763471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.764024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.764208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.783553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:05.785123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:05.798529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:05.866454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 944/1892] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:05.868033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.868313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.868592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.888393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:05.889602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:05.897572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 949/1893] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:05.902268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:05.919484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc1ed3-ad7d-4eac-9732-200dcd76152a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:05.924182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:05.924895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.925174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.925441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.926208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:05.926508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:05.928158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:05.940755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 945/1894] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:05.945321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:05.956077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae468ce-b671-499d-a2e6-f1c8cc42d63f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:05.960714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:05.961382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:05.961664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:05.961937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:05.962632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:05.965092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:05.994803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 950/1895] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:05 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:05.996512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.006697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497196d3-2de4-42c9-acb0-cc076589d660 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.063758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.064432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.064710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.064980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.065670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.067653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.089626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 946/1896] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:05 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:06.093899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.104524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009107f-6cbd-432a-8bfa-dc73fa921c44 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.109549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.110923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.111210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.111482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.112218 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.113824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.124824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 951/1897] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:06.129332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.140394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85d264-2904-4bae-bb3e-0705173ef770 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.146174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a51ccb-4637-4be6-855d-89c7c8191174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.146881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a51ccb-4637-4be6-855d-89c7c8191174 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.147169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a51ccb-4637-4be6-855d-89c7c8191174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.147441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a51ccb-4637-4be6-855d-89c7c8191174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.148174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a51ccb-4637-4be6-855d-89c7c8191174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.149787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a51ccb-4637-4be6-855d-89c7c8191174 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.160519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 947/1898] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:06.164852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.175474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efabcef-3fae-4517-a98f-dd916cb15bbc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.180664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.181547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.181830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.182104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] PATH_INFO: `/v3/projects/93a698b687c64353b795948496851140/users/1f43f8aabc4b4512b528a5e2d82a085d/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.197654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a698b687c64353b795948496851140, user_id=1f43f8aabc4b4512b528a5e2d82a085d, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.201028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.218852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 952/1899] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:06 2022] PUT /identity/v3/projects/93a698b687c64353b795948496851140/users/1f43f8aabc4b4512b528a5e2d82a085d/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:06.229599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.229912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.230182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.258189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] MFA Rules not processed for user `1f43f8aabc4b4512b528a5e2d82a085d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:06.259407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:06.326459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 948/1900] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:06.331108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.394101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:06.398690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22503cfa-c50b-4d1a-8705-5f8292f3e61b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.405072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.405754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.406092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.406374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.407101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.409484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.444423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 953/1901] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:06 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:06.448918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.476771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfb4a3-dc33-4342-aba2-fa447a960201 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.508583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:06.600537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.602781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.603234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.603654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.604530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.606861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.642126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 949/1902] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:06 2022] POST /identity/v3/users => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:06.643207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.654392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd154569-1614-4086-b8e5-aabeb7157b92 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.659290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.660082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.660447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.660781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.661603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.663330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.679377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 954/1903] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:41:06.680628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.694064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c3163-f86a-4fce-9793-0e53aa911d87 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.701559 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.702811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.703315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.703722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] PATH_INFO: `/v3/projects/925ad3a6cd6f476ca48b58ed1a199dd9/users/24b993d4b43e4798bac864827d541e42/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.724618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=925ad3a6cd6f476ca48b58ed1a199dd9, user_id=24b993d4b43e4798bac864827d541e42, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.728026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.743774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 950/1904] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:06 2022] PUT /identity/v3/projects/925ad3a6cd6f476ca48b58ed1a199dd9/users/24b993d4b43e4798bac864827d541e42/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:06.747930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.758636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186480f4-5910-4fe2-9f45-f9165f79bf84 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.819641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a28cc-c6bf-4b41-9adb-1f12399959ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.820325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a28cc-c6bf-4b41-9adb-1f12399959ce admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.820603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a28cc-c6bf-4b41-9adb-1f12399959ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.820883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a28cc-c6bf-4b41-9adb-1f12399959ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.821749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a28cc-c6bf-4b41-9adb-1f12399959ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.823403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a28cc-c6bf-4b41-9adb-1f12399959ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.834132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 955/1905] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:06.839570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:06.850705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33c29e-04ae-4a7d-bd85-e909201fa80c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:06.855582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:06.858602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.859117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.859807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] PATH_INFO: `/v3/projects/925ad3a6cd6f476ca48b58ed1a199dd9/users/24b993d4b43e4798bac864827d541e42/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.862419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925ad3a6cd6f476ca48b58ed1a199dd9, user_id=24b993d4b43e4798bac864827d541e42, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:06.866873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:06.880265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 951/1906] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:06 2022] PUT /identity/v3/projects/925ad3a6cd6f476ca48b58ed1a199dd9/users/24b993d4b43e4798bac864827d541e42/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:06.891803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:06.892088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:06.892438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:06.923922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] MFA Rules not processed for user `24b993d4b43e4798bac864827d541e42`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:06.925452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:07.013348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 956/1907] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:07.023848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.025797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:07.103902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592e90-5d9c-4249-a1bd-752b12db4a29 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.110620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fb3cf-da98-490b-a857-ccd437405ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.111632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fb3cf-da98-490b-a857-ccd437405ccb service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.112148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fb3cf-da98-490b-a857-ccd437405ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.112616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fb3cf-da98-490b-a857-ccd437405ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.199946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fb3cf-da98-490b-a857-ccd437405ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:07.201851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fb3cf-da98-490b-a857-ccd437405ccb service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:07.214335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 952/1908] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:41:07.622945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.624857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:07.680435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:07.694337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc1807-495b-45ba-b48b-bb4960565b0f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.698934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.699971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.700191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.700459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] PATH_INFO: `/v3/users/b2b28dcf77bf4325a5a6d983c579f68c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.701689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b28dcf77bf4325a5a6d983c579f68c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:07.703669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:07.750997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b28dcf77bf4325a5a6d983c579f68c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:07.751622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b28dcf77bf4325a5a6d983c579f68c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:07.758568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b28dcf77bf4325a5a6d983c579f68c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:07.759203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b28dcf77bf4325a5a6d983c579f68c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:07.770918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b28dcf77bf4325a5a6d983c579f68c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:07.786897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 957/1909] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:07 2022] DELETE /identity/v3/users/b2b28dcf77bf4325a5a6d983c579f68c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:07.791583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.791867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.792129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.810495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:07.811885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:07.826336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:07.876908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 953/1910] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:07.887217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:07.889045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:07.950499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a80b71-a2ce-4641-a6f2-710992e8cd13 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:07.954525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:07.955222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:07.955495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:07.955745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:07.965306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:07.967032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:07.977298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 958/1911] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:08.346215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.347877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:08.365347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ed89f-a32b-465a-a77d-7ab078eccf68 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.379798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.380956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.386189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.386471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] PATH_INFO: `/v3/projects/4877ecc00a454c4db997977468cb7e30` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.388824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4877ecc00a454c4db997977468cb7e30)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:08.390956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:08.471316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4877ecc00a454c4db997977468cb7e30', 'request_id': 'req-08468615-a70a-41f8-ac42-fbc71f78a752'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.480072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4877ecc00a454c4db997977468cb7e30', 'request_id': 'req-08468615-a70a-41f8-ac42-fbc71f78a752'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.489028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4877ecc00a454c4db997977468cb7e30', 'request_id': 'req-08468615-a70a-41f8-ac42-fbc71f78a752'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.507882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] The token cache is being invalidate because project 4877ecc00a454c4db997977468cb7e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:08.508165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.508774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.510348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 954/1912] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:08 2022] DELETE /identity/v3/projects/4877ecc00a454c4db997977468cb7e30 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:08.514346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.524293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ada08-9f62-45aa-b01a-e82a3b549308 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:08.580091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:08.581018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:08.581269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:08.581531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] PATH_INFO: `/v3/users/41c19550924e4bae84544fd41652b93c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:08.582860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c19550924e4bae84544fd41652b93c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:08.584777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:08.603155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:08.640624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c19550924e4bae84544fd41652b93c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.641315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c19550924e4bae84544fd41652b93c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.648265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c19550924e4bae84544fd41652b93c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.648876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c19550924e4bae84544fd41652b93c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.666818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c19550924e4bae84544fd41652b93c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:08.684658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 959/1913] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:08 2022] DELETE /identity/v3/users/41c19550924e4bae84544fd41652b93c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:08.970008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:08.972202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:08.987109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08468615-a70a-41f8-ac42-fbc71f78a752 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.045714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.046558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.046873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.047159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] PATH_INFO: `/v3/projects/b1a56f14b09e44429b9126694fed554a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.048351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a56f14b09e44429b9126694fed554a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.051498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.103297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.106215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a56f14b09e44429b9126694fed554a', 'request_id': 'req-a0b1a20f-355d-4ef3-a442-147d04ba4757'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.113540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a56f14b09e44429b9126694fed554a', 'request_id': 'req-a0b1a20f-355d-4ef3-a442-147d04ba4757'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.119341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a56f14b09e44429b9126694fed554a', 'request_id': 'req-a0b1a20f-355d-4ef3-a442-147d04ba4757'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.134610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] The token cache is being invalidate because project b1a56f14b09e44429b9126694fed554a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:09.135172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.135788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.137460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 955/1914] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:08 2022] DELETE /identity/v3/projects/b1a56f14b09e44429b9126694fed554a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:09.141949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.143392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:09.157399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0f192-5309-480d-9e6d-c898a3b1dbd1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.215452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.216317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.216596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.216865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] PATH_INFO: `/v3/users/262e96651a604ed7afbf98219e8d9458` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.218063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=262e96651a604ed7afbf98219e8d9458)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.220047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.272842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262e96651a604ed7afbf98219e8d9458'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.273523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262e96651a604ed7afbf98219e8d9458'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.282260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262e96651a604ed7afbf98219e8d9458'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.282995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262e96651a604ed7afbf98219e8d9458'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.297603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4f3a60e-3ad4-49cf-9fc8-bb4fcfb44cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262e96651a604ed7afbf98219e8d9458'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.330877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 960/1915] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:09 2022] DELETE /identity/v3/users/262e96651a604ed7afbf98219e8d9458 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:09.680979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.682553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:09.696953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1a20f-355d-4ef3-a442-147d04ba4757 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.761975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.762940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.763324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.763600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] PATH_INFO: `/v3/projects/7969ad6e7ad14636989adc92d768be3f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.764848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7969ad6e7ad14636989adc92d768be3f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.766829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:09.817043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7969ad6e7ad14636989adc92d768be3f', 'request_id': 'req-0d6246f4-b8f6-493d-b3ba-199a762de3c6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.823465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7969ad6e7ad14636989adc92d768be3f', 'request_id': 'req-0d6246f4-b8f6-493d-b3ba-199a762de3c6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.829098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7969ad6e7ad14636989adc92d768be3f', 'request_id': 'req-0d6246f4-b8f6-493d-b3ba-199a762de3c6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.843038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] The token cache is being invalidate because project 7969ad6e7ad14636989adc92d768be3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:09.843425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.844052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0d6246f4-b8f6-493d-b3ba-199a762de3c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:09.845825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 956/1916] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:09 2022] DELETE /identity/v3/projects/7969ad6e7ad14636989adc92d768be3f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:09.851442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.851943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.852032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.870104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:09.871358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:09.880609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:09.933006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 961/1917] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:09.942048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.942461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.942735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.961677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:09.962910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:09.971966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 957/1918] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:09.976302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:09.987181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8868b-8e08-4198-8d7f-d2192f1917b8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:09.992286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:09.992944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:09.993230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:09.993507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:09.994244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:09.994549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:09.996399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.008457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 962/1919] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:10.013356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.023076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a308446-b1f4-490c-b205-eab03d96d21d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.028746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.029402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.029676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.029941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.030671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.032709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.062488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 958/1920] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:10 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:10.066913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.077481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76524-9cb7-4517-9c1e-81386c1f1d5d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.146876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.147705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.148007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.148307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.149244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.152496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.184332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 963/1921] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:10 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:10.188119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.198788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829934d-ce18-4c12-b392-8c12786cea87 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.203797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.204451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.204721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.204988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.205708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.207347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.218418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 959/1922] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:10.222550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.233070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78944286-d544-4bb9-902f-204ea40097b0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.238400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966b7e5-c141-4b33-bf45-c8b65e3298da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.239093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966b7e5-c141-4b33-bf45-c8b65e3298da admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.239368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966b7e5-c141-4b33-bf45-c8b65e3298da admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.239637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966b7e5-c141-4b33-bf45-c8b65e3298da admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.240357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966b7e5-c141-4b33-bf45-c8b65e3298da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.241939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966b7e5-c141-4b33-bf45-c8b65e3298da admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.252520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 964/1923] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:10.257257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.272777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd668975-72b8-464f-9595-cb0572be25fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.279809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.280698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.280978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.281251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] PATH_INFO: `/v3/projects/e69c90f824c141ce9474b12fe92b64b9/users/4ad3abdff14941d4be67d33ad3f8d37b/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.301092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69c90f824c141ce9474b12fe92b64b9, user_id=4ad3abdff14941d4be67d33ad3f8d37b, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.305514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.319951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.325523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 960/1924] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:10 2022] PUT /identity/v3/projects/e69c90f824c141ce9474b12fe92b64b9/users/4ad3abdff14941d4be67d33ad3f8d37b/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:10.336412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.336533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.336758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.364884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] MFA Rules not processed for user `4ad3abdff14941d4be67d33ad3f8d37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:10.366118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:10.447127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 965/1925] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:10.457730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.459666 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:10.525432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb4b4b-ee30-46b5-8043-0f93e90121ce admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.535691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918481bd-1d6c-4db8-b440-d8a625fc1db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.536691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918481bd-1d6c-4db8-b440-d8a625fc1db9 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.536979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918481bd-1d6c-4db8-b440-d8a625fc1db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.537253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918481bd-1d6c-4db8-b440-d8a625fc1db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.549792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918481bd-1d6c-4db8-b440-d8a625fc1db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.551369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918481bd-1d6c-4db8-b440-d8a625fc1db9 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.564897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 961/1926] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:10.589535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.599731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:10.653747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11882e-d819-493d-9f93-ac865020c839 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.658854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.659728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.660006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.660274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] PATH_INFO: `/v3/users/4ad3abdff14941d4be67d33ad3f8d37b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.661457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad3abdff14941d4be67d33ad3f8d37b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.663429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.714200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad3abdff14941d4be67d33ad3f8d37b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:10.714901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad3abdff14941d4be67d33ad3f8d37b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:10.727826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad3abdff14941d4be67d33ad3f8d37b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:10.728647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad3abdff14941d4be67d33ad3f8d37b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:10.749566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad3abdff14941d4be67d33ad3f8d37b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:10.770283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 966/1927] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:10 2022] DELETE /identity/v3/users/4ad3abdff14941d4be67d33ad3f8d37b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:10.774907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.775332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.775994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.807829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:10.809864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:10.885569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 962/1928] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:41:10.895318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:10.896761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:10.955158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1c89-c940-4fe9-8561-cd57dde44556 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:10.959368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76212698-5a60-453e-916c-27c841762d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:10.960022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212698-5a60-453e-916c-27c841762d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:10.960303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212698-5a60-453e-916c-27c841762d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:10.960566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212698-5a60-453e-916c-27c841762d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:10.969981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76212698-5a60-453e-916c-27c841762d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:10.971452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76212698-5a60-453e-916c-27c841762d47 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:10.980973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 967/1929] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:11.276961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.278875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:11.293662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a814-1570-4b1c-adca-8dd669bafb26 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.299959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.300833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.301099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.301361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] PATH_INFO: `/v3/projects/e69c90f824c141ce9474b12fe92b64b9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.302691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69c90f824c141ce9474b12fe92b64b9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.304684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.360035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69c90f824c141ce9474b12fe92b64b9', 'request_id': 'req-a8c8319a-24e3-4386-9d15-de18988da68e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:11.370971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69c90f824c141ce9474b12fe92b64b9', 'request_id': 'req-a8c8319a-24e3-4386-9d15-de18988da68e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:11.377608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69c90f824c141ce9474b12fe92b64b9', 'request_id': 'req-a8c8319a-24e3-4386-9d15-de18988da68e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:11.392002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] The token cache is being invalidate because project e69c90f824c141ce9474b12fe92b64b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:11.392309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:11.392927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a8c8319a-24e3-4386-9d15-de18988da68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:11.394530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 963/1930] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:11 2022] DELETE /identity/v3/projects/e69c90f824c141ce9474b12fe92b64b9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:11.399619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.399907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.400178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.419811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:11.421040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:11.450405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.497183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 968/1931] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:41:11.507601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.507939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.508308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.528315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:11.529544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:11.538393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 964/1932] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:11.543309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.555485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda9278-b046-4eb5-aa00-ce6631cdc5fa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.561585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.562445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.562730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.563041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.564295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:11.566012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.566216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.580699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 969/1933] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:11.585377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.607442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48194f-1ccb-45ea-bd24-e5958fc31104 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.612463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.613203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.613547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.613861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.615820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.620526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.648169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:11.658951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 965/1934] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:11 2022] POST /identity/v3/projects => generated 397 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:11.663120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.677106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0f360-6fcc-41a5-9d72-c129786b9d6a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.749587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.750271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.750548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.750839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.751558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.753838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.775796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 970/1935] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:11 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:11.783934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.794993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452417ed-7df5-43b2-ba40-ca550a958cd1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.801376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.801509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.802180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.802180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.802720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.804446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.816918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 966/1936] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:11.826219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.836868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac257dfd-31dd-46b7-8f86-d5757b6d6ae0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.842733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9dde8-62e1-4cea-bd16-79a37f970cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.843528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9dde8-62e1-4cea-bd16-79a37f970cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.843814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9dde8-62e1-4cea-bd16-79a37f970cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.844143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9dde8-62e1-4cea-bd16-79a37f970cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.844842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9dde8-62e1-4cea-bd16-79a37f970cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.846458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9dde8-62e1-4cea-bd16-79a37f970cc1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.857253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 971/1937] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:41:11.861629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:11.871831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd30f00-9ebb-4d39-ac0a-e64c9e0e1393 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:11.876540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:11.877430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.877710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.877981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] PATH_INFO: `/v3/projects/86e4fa4014b3416883a4e31f8b9b10bc/users/52d3eaa68c654b3bba84f4a3923869a5/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.892985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e4fa4014b3416883a4e31f8b9b10bc, user_id=52d3eaa68c654b3bba84f4a3923869a5, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:11.895723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:11.914796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 967/1938] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:11 2022] PUT /identity/v3/projects/86e4fa4014b3416883a4e31f8b9b10bc/users/52d3eaa68c654b3bba84f4a3923869a5/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:11.922893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:11.923379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:11.923453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:11.954725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] MFA Rules not processed for user `52d3eaa68c654b3bba84f4a3923869a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:11.955976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:12.035558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 972/1939] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:12.047279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:12.048974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:12.107339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69a387-149f-4f8d-913c-9e54d84e0e3a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:12.112247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:12.113091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:12.113370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:12.113645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:12.123490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:12.124973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:12.134121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 968/1940] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:22.032522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.087762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:22.090139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:22.102852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c444d-0355-4524-801a-6ab1e7a99bea None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:22.106792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:22.107470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:22.107750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:22.108018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:22.109929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:22.129831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:22.181384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:22.182765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bdc13-d841-4fdd-be7d-c6071ac8b9b2 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:22.198791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 973/1941] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:23.535897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:23.608363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a271af-b0b5-4cb4-a682-34e25cfcf06a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:23.612836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:23.613721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:23.613998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:23.614264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] PATH_INFO: `/v3/users/52d3eaa68c654b3bba84f4a3923869a5` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:23.615492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d3eaa68c654b3bba84f4a3923869a5)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:23.617415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:23.697036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d3eaa68c654b3bba84f4a3923869a5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:23.697785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d3eaa68c654b3bba84f4a3923869a5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:23.698432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d3eaa68c654b3bba84f4a3923869a5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:23.711310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d3eaa68c654b3bba84f4a3923869a5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:23.718997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d3eaa68c654b3bba84f4a3923869a5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:23.734259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 969/1942] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:23 2022] DELETE /identity/v3/users/52d3eaa68c654b3bba84f4a3923869a5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:23.738950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:23.739425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:23.739501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:23.757269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:23.758967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:23.842295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 974/1943] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:23.852600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:23.854043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:23.927634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf825a-388a-4481-90c0-4d28b916cc17 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:23.930992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:23.931655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:23.931931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:23.932326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:23.945087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:23.946693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:23.957601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 970/1944] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:41:24.487755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:24.489398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:24.502632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833ca15-165e-4110-96d8-30ab9682cb0e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:24.508680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:24.509560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:24.509839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:24.510109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] PATH_INFO: `/v3/projects/86e4fa4014b3416883a4e31f8b9b10bc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:24.511393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e4fa4014b3416883a4e31f8b9b10bc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:24.513362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:24.564352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e4fa4014b3416883a4e31f8b9b10bc', 'request_id': 'req-7b14de2f-d332-41d1-a123-3b1a8ea5443e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:24.570630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e4fa4014b3416883a4e31f8b9b10bc', 'request_id': 'req-7b14de2f-d332-41d1-a123-3b1a8ea5443e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:24.576566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e4fa4014b3416883a4e31f8b9b10bc', 'request_id': 'req-7b14de2f-d332-41d1-a123-3b1a8ea5443e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:24.590837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] The token cache is being invalidate because project 86e4fa4014b3416883a4e31f8b9b10bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:24.590984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:24.591594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7b14de2f-d332-41d1-a123-3b1a8ea5443e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:24.593269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 975/1945] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:24 2022] DELETE /identity/v3/projects/86e4fa4014b3416883a4e31f8b9b10bc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:41:32.753617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:32.755117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:32.777026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1f66-21fe-41f7-9273-03e5ead72137 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:32.807926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:32.883788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:32.884968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:32.886633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:32.886633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] PATH_INFO: `/v3/users/1f43f8aabc4b4512b528a5e2d82a085d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:32.887598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f43f8aabc4b4512b528a5e2d82a085d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:32.893645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:32.981293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f43f8aabc4b4512b528a5e2d82a085d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:32.982228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f43f8aabc4b4512b528a5e2d82a085d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:32.982883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f43f8aabc4b4512b528a5e2d82a085d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:32.999494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f43f8aabc4b4512b528a5e2d82a085d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:33.008304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f43f8aabc4b4512b528a5e2d82a085d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:33.030611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 971/1946] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:32 2022] DELETE /identity/v3/users/1f43f8aabc4b4512b528a5e2d82a085d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:33.035065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.035588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.035588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.055341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:33.056581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:33.070852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:33.126938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 976/1947] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:33.140796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:33.142304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:33.209888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0eb46-0578-4f0f-a959-1a81485afea5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:33.214773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:33.217146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:33.217455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:33.217730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:33.228676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:33.230703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:33.241485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 972/1948] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:34.052390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.054152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:34.068197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ae12e-8bf3-4ee6-9f76-69eb01351c5f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.073831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.074734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.075499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.075783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] PATH_INFO: `/v3/users/c64343bf0fd44781a687fe0fa866ff43` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.080078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64343bf0fd44781a687fe0fa866ff43)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.080422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.081856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:34.082217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.095154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd2256-1248-40f6-a556-f8206922f404 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.100052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.100932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.101209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.101480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] PATH_INFO: `/v3/projects/93a698b687c64353b795948496851140` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.102693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a698b687c64353b795948496851140)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.104685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.162421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.180397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64343bf0fd44781a687fe0fa866ff43'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.182873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64343bf0fd44781a687fe0fa866ff43'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.184173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a698b687c64353b795948496851140', 'request_id': 'req-943d6f27-c82f-4346-99e1-830e924c886c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.193494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a698b687c64353b795948496851140', 'request_id': 'req-943d6f27-c82f-4346-99e1-830e924c886c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.196582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64343bf0fd44781a687fe0fa866ff43'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.197351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64343bf0fd44781a687fe0fa866ff43'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.209729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a698b687c64353b795948496851140', 'request_id': 'req-943d6f27-c82f-4346-99e1-830e924c886c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.232390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] The token cache is being invalidate because project 93a698b687c64353b795948496851140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:34.232781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.233572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.237206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 973/1949] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:34 2022] DELETE /identity/v3/projects/93a698b687c64353b795948496851140 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:34.240695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28e508ab-2876-4d6c-8a5e-367ee937a137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64343bf0fd44781a687fe0fa866ff43'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.241939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.246097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:34.270220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 977/1950] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:34 2022] DELETE /identity/v3/users/c64343bf0fd44781a687fe0fa866ff43 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:34.277254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.277541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.277816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.317729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:34.319262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:34.372506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d6f27-c82f-4346-99e1-830e924c886c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.404208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 978/1951] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:34.420299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:34.422220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:34.493124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.525831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.526819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.527261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.527630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] PATH_INFO: `/v3/users/24b993d4b43e4798bac864827d541e42` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.529842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa9a5e-c08b-48d6-bc58-d59ab8e8428b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:34.530726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b993d4b43e4798bac864827d541e42)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.532803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.535049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:34.535884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:34.536432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:34.536852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:34.552546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:34.555427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:34.570087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 979/1952] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:41:34.641179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b993d4b43e4798bac864827d541e42'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.642585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b993d4b43e4798bac864827d541e42'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.643903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b993d4b43e4798bac864827d541e42'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.671775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b993d4b43e4798bac864827d541e42'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.697587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-992ce1b5-6a98-4a2c-9cc6-b591adf4653c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b993d4b43e4798bac864827d541e42'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:34.703382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:34.735116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 974/1953] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:34 2022] DELETE /identity/v3/users/24b993d4b43e4798bac864827d541e42 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:35.195846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.212921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844117c5-9520-4733-af97-decd878f82ca service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.325671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.326793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.327082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.327352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] PATH_INFO: `/v3/projects/925ad3a6cd6f476ca48b58ed1a199dd9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.328797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=925ad3a6cd6f476ca48b58ed1a199dd9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.330891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.395929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925ad3a6cd6f476ca48b58ed1a199dd9', 'request_id': 'req-905105f7-5ea1-46c9-a63e-f837a8f4d244'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.403840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925ad3a6cd6f476ca48b58ed1a199dd9', 'request_id': 'req-905105f7-5ea1-46c9-a63e-f837a8f4d244'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.410568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925ad3a6cd6f476ca48b58ed1a199dd9', 'request_id': 'req-905105f7-5ea1-46c9-a63e-f837a8f4d244'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.427306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] The token cache is being invalidate because project 925ad3a6cd6f476ca48b58ed1a199dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:35.427692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.428737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.435583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 980/1954] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:35 2022] DELETE /identity/v3/projects/925ad3a6cd6f476ca48b58ed1a199dd9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:35.437723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485651f-1aa2-417a-a3b2-bcc7b42f31ad None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.438163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485651f-1aa2-417a-a3b2-bcc7b42f31ad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.438428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485651f-1aa2-417a-a3b2-bcc7b42f31ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.450568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.455374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:35.462092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1485651f-1aa2-417a-a3b2-bcc7b42f31ad None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:35.482316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1485651f-1aa2-417a-a3b2-bcc7b42f31ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:35.544044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905105f7-5ea1-46c9-a63e-f837a8f4d244 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.548844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 975/1955] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:35.551249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.552633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.553103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.553691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] PATH_INFO: `/v3/projects/155ebb991459454686c07cbc8adc3b69` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.555316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155ebb991459454686c07cbc8adc3b69)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.557083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.557371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.557611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.557779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.566999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.580586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:35.582010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:35.591444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 976/1956] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:35.595654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.609144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4e996-9c76-4d5b-8b96-8959845654fe None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.627357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155ebb991459454686c07cbc8adc3b69', 'request_id': 'req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.636290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155ebb991459454686c07cbc8adc3b69', 'request_id': 'req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.641489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155ebb991459454686c07cbc8adc3b69', 'request_id': 'req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.663077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] The token cache is being invalidate because project 155ebb991459454686c07cbc8adc3b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:35.663665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.664496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.666364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 981/1957] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:35 2022] DELETE /identity/v3/projects/155ebb991459454686c07cbc8adc3b69 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:41:35.670872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.673365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:35.690739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.691903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.692312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.692639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.693387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:35.693698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.696279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.711945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 977/1958] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:41:35.716764 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.719492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:35.738251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acc141-af2e-44d4-85df-fe4d7059c6dc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.749499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.750196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.750478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.750823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.751728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.753750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.777384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069dcb0-d5c0-47f4-9417-341d5bf7a4e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.783734 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.785776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.787595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.788040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] PATH_INFO: `/v3/users/79a2218682c84ddda521885588d4cc60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.789420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a2218682c84ddda521885588d4cc60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.791567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.796235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 978/1959] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:35 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:35.809440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.880088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a33f7-4739-466a-b109-b2cb53a8542c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:35.881971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a2218682c84ddda521885588d4cc60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.882686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a2218682c84ddda521885588d4cc60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.888399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:35.889085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:35.889364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:35.889633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:35.890328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:35.892471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:35.896257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a2218682c84ddda521885588d4cc60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.897199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a2218682c84ddda521885588d4cc60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.901447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:35.912219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a2218682c84ddda521885588d4cc60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:35.921562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 979/1960] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:35 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:35.932749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:35.935854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:35.938996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 982/1961] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:35 2022] DELETE /identity/v3/users/79a2218682c84ddda521885588d4cc60 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:35.953011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a3cb2-54a0-4f17-9114-e383f621ca7a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.022503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.023228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.023511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.023788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.024522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.026626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.037492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 980/1962] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:36.041923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.060811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230297a-c909-4b8e-8d0a-f99708384f79 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.075902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a32fe-2433-409c-bad8-5579e9391302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.077073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a32fe-2433-409c-bad8-5579e9391302 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.077547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a32fe-2433-409c-bad8-5579e9391302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.078081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a32fe-2433-409c-bad8-5579e9391302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.079101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a32fe-2433-409c-bad8-5579e9391302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.081055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a32fe-2433-409c-bad8-5579e9391302 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.095143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 983/1963] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:36.101084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.113333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7452cf-cdd8-46b5-8228-e71c3cb6bd29 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.118659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.119668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.119933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.120181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] PATH_INFO: `/v3/projects/4b05021ee75a47f29be880e447488dcb/users/870b2f691e0145a6a28e541ad506af02/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.140016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b05021ee75a47f29be880e447488dcb, user_id=870b2f691e0145a6a28e541ad506af02, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.144552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.166999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 981/1964] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:36 2022] PUT /identity/v3/projects/4b05021ee75a47f29be880e447488dcb/users/870b2f691e0145a6a28e541ad506af02/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:36.171235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.171235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.171235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.177939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.196467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:36.199344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:36.255630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.257982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:36.271207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 984/1965] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:36.280189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.281420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:36.341870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3fe48-fa83-4fba-8285-907d8b47859b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.346077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b13ec-6745-47c7-8770-b8ecf7ed7c55 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.347972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.349004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.349284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.349595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] PATH_INFO: `/v3/projects/f89aea4c147c4df69e4a29357cc2a998` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.350181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05201e-9e17-4887-b43d-3e8041c2a9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.350738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f89aea4c147c4df69e4a29357cc2a998)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.350988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05201e-9e17-4887-b43d-3e8041c2a9ae service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.351329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05201e-9e17-4887-b43d-3e8041c2a9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.351664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05201e-9e17-4887-b43d-3e8041c2a9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.352788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.362821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05201e-9e17-4887-b43d-3e8041c2a9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.364352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05201e-9e17-4887-b43d-3e8041c2a9ae service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.375635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 985/1966] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:36.414163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f89aea4c147c4df69e4a29357cc2a998', 'request_id': 'req-c1a884c6-19cd-4484-88bd-3f2a11351c2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:36.422977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f89aea4c147c4df69e4a29357cc2a998', 'request_id': 'req-c1a884c6-19cd-4484-88bd-3f2a11351c2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:36.429742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f89aea4c147c4df69e4a29357cc2a998', 'request_id': 'req-c1a884c6-19cd-4484-88bd-3f2a11351c2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:36.431966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=4,keystone:SELECT=90,keystone:DELETE=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.449511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] The token cache is being invalidate because project f89aea4c147c4df69e4a29357cc2a998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:36.451163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:36.452471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c1a884c6-19cd-4484-88bd-3f2a11351c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:36.455132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 982/1967] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:36 2022] DELETE /identity/v3/projects/f89aea4c147c4df69e4a29357cc2a998 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:41:36.460584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.460919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.461149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.481124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:36.482407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:36.551098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 986/1968] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:36.557058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.557457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.557640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.578930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:36.580534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:36.589421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 983/1969] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:36.593756 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.605060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb102a0-82c0-4f65-90f2-8d19690d4597 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.610641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.611371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.611654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.611928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.612748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:36.613058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.615214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.632596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.641746 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 987/1970] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:36.646054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a160f-10a1-49ee-9679-0a7daa3cb801 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.652343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.653023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.653318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.653565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.654264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.656273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.685703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 984/1971] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:36 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:36.690426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.703543 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fecb5b-6659-4efd-a499-0ed90e0942f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.788750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:36.838124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.850779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.851768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.851921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.855265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.861634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.901444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 988/1972] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:36 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:36.905519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.919832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86792be2-d49e-4897-9890-16b1861de218 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.925912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.927186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.927186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.927956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.928174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.930412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.950485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 985/1973] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:36.952403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:36.965159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af206097-643a-4eac-84ea-6a9d677a56e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:36.973471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfac2e-8de4-430c-8b2b-d5b50675d831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:36.974516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfac2e-8de4-430c-8b2b-d5b50675d831 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:36.974913 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfac2e-8de4-430c-8b2b-d5b50675d831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:36.975346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfac2e-8de4-430c-8b2b-d5b50675d831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:36.976123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfac2e-8de4-430c-8b2b-d5b50675d831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:36.977858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfac2e-8de4-430c-8b2b-d5b50675d831 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:36.992440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 989/1974] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:36.996637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.008727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69359eeb-efc5-4eaf-8e9d-40b279701284 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.014106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.015059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.015341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.015614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] PATH_INFO: `/v3/projects/6060a054812b40b3bf0af8d2ba6277cd/users/047f6cc1357141f59273c5fee0699c0b/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.032202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6060a054812b40b3bf0af8d2ba6277cd, user_id=047f6cc1357141f59273c5fee0699c0b, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.035273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.052181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 986/1975] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:36 2022] PUT /identity/v3/projects/6060a054812b40b3bf0af8d2ba6277cd/users/047f6cc1357141f59273c5fee0699c0b/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:37.063125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b488e4-828c-43a1-9484-e2a0a22a7d9f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.063495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b488e4-828c-43a1-9484-e2a0a22a7d9f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.063781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b488e4-828c-43a1-9484-e2a0a22a7d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.095698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f0b488e4-828c-43a1-9484-e2a0a22a7d9f None None] MFA Rules not processed for user `047f6cc1357141f59273c5fee0699c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:37.096952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0b488e4-828c-43a1-9484-e2a0a22a7d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:37.172217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 990/1976] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:37.173892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.194534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.194920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.195192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.224632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] MFA Rules not processed for user `870b2f691e0145a6a28e541ad506af02`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:37.225999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:37.241865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87a6eb-9526-4857-9567-471239bf4d7f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.247587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.248496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.248796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.249491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.249761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.251717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.290919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 987/1977] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:37 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:37.293289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.298318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 991/1978] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:37.310091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.312044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:37.352944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.364758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e95654-d23b-41fa-8b98-ad822a6445ed admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.370353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.371098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.371377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.371648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.372509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.374534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.382957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3e7ef-7514-40a5-89ad-51605aa61539 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.402784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 988/1979] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:37 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:37.404864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.410285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=7,keystone:DELETE=1,keystone:SELECT=92 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:37.416599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba7e0-59b7-41f6-921c-e8246ebbcf7d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.423080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.424439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.425797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.425797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.426077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.428423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.443327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 989/1980] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:37.444444 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.445097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.445863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.446266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.446632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.455930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c51933-8fa1-48c4-8eb2-d4507af086a0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.460879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.461839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.462070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.462334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] PATH_INFO: `/v3/projects/3aea78dc15374ae2bd80bf9acab9af94/users/7cd74f25a11844759915a8aa171825b7/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.478522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aea78dc15374ae2bd80bf9acab9af94, user_id=7cd74f25a11844759915a8aa171825b7, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.481300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.502852 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 990/1981] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:37 2022] PUT /identity/v3/projects/3aea78dc15374ae2bd80bf9acab9af94/users/7cd74f25a11844759915a8aa171825b7/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:41:37.503805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.512506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.514495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.527248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 992/1982] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:41:37.569766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b48202-e3b6-479b-8818-b421d82a9296 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.574909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64314b6c-b5a5-4022-b1fa-80903c8f77fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.575817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64314b6c-b5a5-4022-b1fa-80903c8f77fc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.576101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64314b6c-b5a5-4022-b1fa-80903c8f77fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.576372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64314b6c-b5a5-4022-b1fa-80903c8f77fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.577106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64314b6c-b5a5-4022-b1fa-80903c8f77fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.578707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64314b6c-b5a5-4022-b1fa-80903c8f77fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.588988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 991/1983] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:37.593832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.605478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bc66c-a380-45b9-953b-5d9ae583f3a0 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.610653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.611581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.611861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.612161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] PATH_INFO: `/v3/projects/3aea78dc15374ae2bd80bf9acab9af94/users/7cd74f25a11844759915a8aa171825b7/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.614153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aea78dc15374ae2bd80bf9acab9af94, user_id=7cd74f25a11844759915a8aa171825b7, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.616982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.628269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 993/1984] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:37 2022] PUT /identity/v3/projects/3aea78dc15374ae2bd80bf9acab9af94/users/7cd74f25a11844759915a8aa171825b7/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:37.640761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.641050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.641330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.677921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] MFA Rules not processed for user `7cd74f25a11844759915a8aa171825b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:37.679588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:37.760581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 992/1985] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:37.773087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:37.775924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:37.874406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd134a2-998a-4e17-8eef-6f5607072501 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:37.878177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88a7ac-0551-474c-9b03-e6dce70b84cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:37.879114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88a7ac-0551-474c-9b03-e6dce70b84cc service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:37.879393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88a7ac-0551-474c-9b03-e6dce70b84cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:37.879696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88a7ac-0551-474c-9b03-e6dce70b84cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:37.892457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88a7ac-0551-474c-9b03-e6dce70b84cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:37.893900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88a7ac-0551-474c-9b03-e6dce70b84cc service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:37.904436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 994/1986] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:38.333680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.389393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:38.410988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bf1c6-523c-4352-b368-27e86d53440b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.415924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.417009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.417280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.417524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] PATH_INFO: `/v3/users/047f6cc1357141f59273c5fee0699c0b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.418921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=047f6cc1357141f59273c5fee0699c0b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.421002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.475877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047f6cc1357141f59273c5fee0699c0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:38.477064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047f6cc1357141f59273c5fee0699c0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:38.477842 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047f6cc1357141f59273c5fee0699c0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:38.490482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047f6cc1357141f59273c5fee0699c0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:38.497543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047f6cc1357141f59273c5fee0699c0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:38.514102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 993/1987] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:38 2022] DELETE /identity/v3/users/047f6cc1357141f59273c5fee0699c0b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:38.519182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9c46-991c-4672-9bf3-1b8bcb2f9436 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.519742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9c46-991c-4672-9bf3-1b8bcb2f9436 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.530842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9c46-991c-4672-9bf3-1b8bcb2f9436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.550189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2d2d9c46-991c-4672-9bf3-1b8bcb2f9436 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:38.551497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d2d9c46-991c-4672-9bf3-1b8bcb2f9436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:38.578649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:38.616422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 995/1988] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:38.626220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:38.627743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:38.690450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a5c62-b78b-4fe8-9f13-29119c7d8da3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:38.694314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:38.695022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.695305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.695577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.705345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:38.707020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:38.717228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 994/1989] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:38.930969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:38.931335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:38.931622 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:38.951006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] MFA Rules not processed for user `870b2f691e0145a6a28e541ad506af02`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:38.952398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:39.027295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.028787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 996/1990] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:39.031085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:39.050105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d620340-794d-4f0e-8e96-c8dd741a1b49 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.058024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.059280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.059754 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.060205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] PATH_INFO: `/v3/projects/6060a054812b40b3bf0af8d2ba6277cd` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.061787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6060a054812b40b3bf0af8d2ba6277cd)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.064514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.145707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6060a054812b40b3bf0af8d2ba6277cd', 'request_id': 'req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.152940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6060a054812b40b3bf0af8d2ba6277cd', 'request_id': 'req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.159119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6060a054812b40b3bf0af8d2ba6277cd', 'request_id': 'req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.162419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.177067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] The token cache is being invalidate because project 6060a054812b40b3bf0af8d2ba6277cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:39.177352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.177996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.179927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 995/1991] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:39 2022] DELETE /identity/v3/projects/6060a054812b40b3bf0af8d2ba6277cd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:39.184063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.196350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90b0c0-e16f-43f7-ad0e-196e374b35fd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.256227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.257128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.257409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.257683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] PATH_INFO: `/v3/users/7cd74f25a11844759915a8aa171825b7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.259848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd74f25a11844759915a8aa171825b7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.262081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:39.324675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd74f25a11844759915a8aa171825b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.325731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd74f25a11844759915a8aa171825b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.334010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd74f25a11844759915a8aa171825b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.335024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd74f25a11844759915a8aa171825b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.351526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d5b9b645-471a-413d-aba4-bbb07bc447a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd74f25a11844759915a8aa171825b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:39.358515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:39.374737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 997/1992] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:39 2022] DELETE /identity/v3/users/7cd74f25a11844759915a8aa171825b7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:39.875865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:39.877304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:39.891644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d3dfc-db7c-41db-811e-6ec3c1ee2701 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:39.955998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:39.956880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:39.957161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:39.957437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] PATH_INFO: `/v3/projects/3aea78dc15374ae2bd80bf9acab9af94` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:39.958693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aea78dc15374ae2bd80bf9acab9af94)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:39.960639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.020212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aea78dc15374ae2bd80bf9acab9af94', 'request_id': 'req-19a333e2-3435-4583-8fb4-fc942a7e9597'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:40.028780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aea78dc15374ae2bd80bf9acab9af94', 'request_id': 'req-19a333e2-3435-4583-8fb4-fc942a7e9597'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:40.035102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aea78dc15374ae2bd80bf9acab9af94', 'request_id': 'req-19a333e2-3435-4583-8fb4-fc942a7e9597'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:40.053037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] The token cache is being invalidate because project 3aea78dc15374ae2bd80bf9acab9af94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:40.053447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:40.054561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a333e2-3435-4583-8fb4-fc942a7e9597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:40.062496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 996/1993] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:39 2022] DELETE /identity/v3/projects/3aea78dc15374ae2bd80bf9acab9af94 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:40.073698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.074168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.074370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.135340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:40.137620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:40.250661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 998/1994] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:40.256526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ff471-88d5-4459-88d7-f1581281524b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.256526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ff471-88d5-4459-88d7-f1581281524b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.256526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ff471-88d5-4459-88d7-f1581281524b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.287146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-083ff471-88d5-4459-88d7-f1581281524b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:40.288473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-083ff471-88d5-4459-88d7-f1581281524b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:40.313772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 997/1995] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:40.318117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.332921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba7440-0673-435b-a45f-fdfddb3395f4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.339142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.340035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.340478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.340693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.342040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:40.342357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.344180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.377238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ff471-88d5-4459-88d7-f1581281524b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.394782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 999/1996] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:40.396799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ff471-88d5-4459-88d7-f1581281524b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.403220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.404671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.405423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.406586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.407988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.413926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.470001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.479524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 998/1997] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:40.492467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f50442-caa9-425e-9ef9-8de1ce29840f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.611597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.612320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.612604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.612986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.613584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.617673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.653303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.669531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1000/1998] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:40 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:40.674909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb0305-6262-45a2-96e4-956cd9cde641 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.691963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.692266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.692548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.692820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.693614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.698370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.727004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.739268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a24f-949b-4806-8ce4-1e23fcc0f0f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.739609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 999/1999] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:41:40.757609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74bb8a-125d-4168-915f-998ad16830b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.758695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74bb8a-125d-4168-915f-998ad16830b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.759230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74bb8a-125d-4168-915f-998ad16830b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.759730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74bb8a-125d-4168-915f-998ad16830b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.761049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74bb8a-125d-4168-915f-998ad16830b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.767773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74bb8a-125d-4168-915f-998ad16830b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.800418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1001/2000] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:40.801423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:40.818035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f832-131f-45b9-9c3f-0692c5e104ff admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:40.825162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:40.826525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.827022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.827438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] PATH_INFO: `/v3/projects/b7e29703c6204f09a7f14bb7984fa126/users/33badbd917a74b65b9eaf61dc1fefa3d/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.853558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7e29703c6204f09a7f14bb7984fa126, user_id=33badbd917a74b65b9eaf61dc1fefa3d, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:40.856702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:40.880020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1000/2001] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:40 2022] PUT /identity/v3/projects/b7e29703c6204f09a7f14bb7984fa126/users/33badbd917a74b65b9eaf61dc1fefa3d/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:40.891579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:40.891868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:40.892146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:40.937840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] MFA Rules not processed for user `33badbd917a74b65b9eaf61dc1fefa3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:40.939449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:41.044991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.085862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1002/2002] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:41.088982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.132337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:41.197966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9009de-703d-4d2e-a9f7-c0567dcca798 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.219928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.220137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.220290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.220511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.221573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.223286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.265628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1001/2003] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:41 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:41.269761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.285596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c79f8d-033d-4c2b-a914-bff5d15b679e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.360817 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.361521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.361807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.362084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.362829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.365271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.410936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.411151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1003/2004] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:41 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:41:41.428266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2244-8323-48ad-ab9d-7e1e28cf7696 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.436630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.439903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.439903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.440177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.440720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.443247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.459917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1002/2005] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:41:41.461626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.474773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f99d-1ec0-481e-8fa7-a873ac40c4f6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.480923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.481978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.482292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.482589 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] PATH_INFO: `/v3/projects/36f0f238347f4f28a7f821acb2a9fd16/users/7e4e24dd58464c1cb0df6d8fd3a6d5b6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.502863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f0f238347f4f28a7f821acb2a9fd16, user_id=7e4e24dd58464c1cb0df6d8fd3a6d5b6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.505918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.526821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1004/2006] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:41 2022] PUT /identity/v3/projects/36f0f238347f4f28a7f821acb2a9fd16/users/7e4e24dd58464c1cb0df6d8fd3a6d5b6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:41.528346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.546624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ea1cf-b562-4ae5-8769-e93b1a412a36 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.620074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cffee-69ab-4d94-be4d-d9200c3e0e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.620770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cffee-69ab-4d94-be4d-d9200c3e0e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.621048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cffee-69ab-4d94-be4d-d9200c3e0e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.623134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cffee-69ab-4d94-be4d-d9200c3e0e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.623512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cffee-69ab-4d94-be4d-d9200c3e0e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.625486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cffee-69ab-4d94-be4d-d9200c3e0e53 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.635961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1003/2007] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:41.641655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.653134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b65af-778f-41f9-a64c-9f71732fd8b7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.658257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.659411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.659835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.660391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] PATH_INFO: `/v3/projects/36f0f238347f4f28a7f821acb2a9fd16/users/7e4e24dd58464c1cb0df6d8fd3a6d5b6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.662895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f0f238347f4f28a7f821acb2a9fd16, user_id=7e4e24dd58464c1cb0df6d8fd3a6d5b6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:41.666349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:41.696622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.697052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.697329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:41.698839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1005/2008] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:41 2022] PUT /identity/v3/projects/36f0f238347f4f28a7f821acb2a9fd16/users/7e4e24dd58464c1cb0df6d8fd3a6d5b6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:41.729409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] MFA Rules not processed for user `7e4e24dd58464c1cb0df6d8fd3a6d5b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:41.731179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:41.841502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1004/2009] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:41:41.857942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:41.861399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:41.945307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86c87-9852-43b3-aa3c-3724927d1a6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:41.950175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:41.951103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:41.951441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:41.952632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:42.011334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:42.012745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:42.017195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:42.025265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1006/2010] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:45.101757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.104107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:45.126111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:45.179464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5603bcc-0917-45a2-bf29-e3a4ed19d7ab None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.184130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.185017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.185296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.185567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] PATH_INFO: `/v3/users/c00d1d4562584ba69a7a9a5fb016b2ba` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.186825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c00d1d4562584ba69a7a9a5fb016b2ba)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.188851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.238065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c00d1d4562584ba69a7a9a5fb016b2ba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.238727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c00d1d4562584ba69a7a9a5fb016b2ba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.239353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c00d1d4562584ba69a7a9a5fb016b2ba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.260221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c00d1d4562584ba69a7a9a5fb016b2ba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.268454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf022f99-4536-465e-8cbf-dcbaaee57cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c00d1d4562584ba69a7a9a5fb016b2ba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.285284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1005/2011] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:41:45 2022] DELETE /identity/v3/users/c00d1d4562584ba69a7a9a5fb016b2ba => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:45.461719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.473039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e060f5-6b89-4324-bcf5-53afeb6d67d4 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.535500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.536373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.536648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.536915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] PATH_INFO: `/v3/projects/8e6e9d7564324153a7edc2b5f04c4c6c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.538161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6e9d7564324153a7edc2b5f04c4c6c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.540081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.587342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6e9d7564324153a7edc2b5f04c4c6c', 'request_id': 'req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.593224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6e9d7564324153a7edc2b5f04c4c6c', 'request_id': 'req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.599531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6e9d7564324153a7edc2b5f04c4c6c', 'request_id': 'req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.614548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] The token cache is being invalidate because project 8e6e9d7564324153a7edc2b5f04c4c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:41:45.614881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.615526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c77d60a1-e819-4d13-9d1a-bdfbd356d3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:41:45.617100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1007/2012] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:41:45 2022] DELETE /identity/v3/projects/8e6e9d7564324153a7edc2b5f04c4c6c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:45.622214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.622504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.622829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.641645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:45.642898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:45.705377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1006/2013] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:45.713509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d654c-463a-487c-8f80-c2103817099e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.713842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d654c-463a-487c-8f80-c2103817099e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.714107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d654c-463a-487c-8f80-c2103817099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.737322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e83d654c-463a-487c-8f80-c2103817099e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:45.738573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e83d654c-463a-487c-8f80-c2103817099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:45.748549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1008/2014] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:45.753134 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.763815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe60a-5742-4018-9596-7f5b04eebd1e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.773281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.773969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.774249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.774520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.775669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:41:45.775972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.777602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.790692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1007/2015] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:41:45.795278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d654c-463a-487c-8f80-c2103817099e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.805046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d654c-463a-487c-8f80-c2103817099e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.810434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.811122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.811401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.811711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.812502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.814396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.844700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1009/2016] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:41:45 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:45.854602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.859663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb47493-839a-41ea-b109-ca4161704346 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.878538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:45.917871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.918549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.918858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.919132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.919831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.921817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.944946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1008/2017] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:41:45 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:41:45.949003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:45.959735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef38d1-3d35-459a-827d-6083b28afcf6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:45.964588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:45.965488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:45.965773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:45.966052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:45.967264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:45.969007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:45.980213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1010/2018] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:45.985295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.000111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36f604-dc94-4bb3-b14a-b7622cd36c95 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.007271 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd1e8d-3f61-4b38-9869-1ee343bfe0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.008359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd1e8d-3f61-4b38-9869-1ee343bfe0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.008816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd1e8d-3f61-4b38-9869-1ee343bfe0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.009261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd1e8d-3f61-4b38-9869-1ee343bfe0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.010040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd1e8d-3f61-4b38-9869-1ee343bfe0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.012184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd1e8d-3f61-4b38-9869-1ee343bfe0f8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.023433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1009/2019] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:41:46.028763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.039382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b067c-9436-4f1e-bb0a-d23227b6f7b8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.044350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.045234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.045650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.046002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] PATH_INFO: `/v3/projects/b53f565629784cf98c718c356e6dc28b/users/b5d566b4cba14d62be7ad1017e2b9ff9/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.061703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53f565629784cf98c718c356e6dc28b, user_id=b5d566b4cba14d62be7ad1017e2b9ff9, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.064615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.081495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1011/2020] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:41:46 2022] PUT /identity/v3/projects/b53f565629784cf98c718c356e6dc28b/users/b5d566b4cba14d62be7ad1017e2b9ff9/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:41:46.093039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.093465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.093569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.123064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] MFA Rules not processed for user `b5d566b4cba14d62be7ad1017e2b9ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:41:46.124452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:46.196313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1010/2021] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:41:46.207499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:46.209558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:46.241701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:46.275658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c7b5-a282-4442-98b7-64725748141d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:46.279755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:46.280408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:46.280696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:46.280965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:46.290428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:46.291908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:46.302482 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1012/2022] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:48.864329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:41:48.865908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:48.879826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2141e1-6773-4c6f-a4e3-4091169bb42a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:41:48.884563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:41:48.885260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:41:48.885553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:41:48.885814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:41:48.888035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:41:48.960907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:41:48.962394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9422bd-b277-4f75-80eb-f25cb85ed0bc service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:41:48.974062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1011/2023] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:41:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:41:56.298007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=15 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:41:58.968658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:06.516442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.640958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a9694-edd5-496b-b43d-fbb413b5a54e service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:06.648065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:06.649395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.649629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.649867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] PATH_INFO: `/v3/users/b5d566b4cba14d62be7ad1017e2b9ff9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.656949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d566b4cba14d62be7ad1017e2b9ff9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:06.659270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:06.711065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d566b4cba14d62be7ad1017e2b9ff9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:06.711815 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d566b4cba14d62be7ad1017e2b9ff9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:06.718801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d566b4cba14d62be7ad1017e2b9ff9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:06.719490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d566b4cba14d62be7ad1017e2b9ff9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:06.733151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d566b4cba14d62be7ad1017e2b9ff9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:06.747701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1013/2024] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:06 2022] DELETE /identity/v3/users/b5d566b4cba14d62be7ad1017e2b9ff9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:06.752505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:06.753043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:06.753043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:06.805244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:06.806499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:06.873661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1012/2025] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:06.883644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:06.885413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:06.937782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.000442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c6067-cb16-48fa-bca9-a0351abe0e4e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.004697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110d63-355c-488d-9f99-e057d65a9fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.005372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110d63-355c-488d-9f99-e057d65a9fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.005611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110d63-355c-488d-9f99-e057d65a9fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.005875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110d63-355c-488d-9f99-e057d65a9fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.016471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8110d63-355c-488d-9f99-e057d65a9fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.017937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8110d63-355c-488d-9f99-e057d65a9fa6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.030770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1014/2026] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:07.346602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.348571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:07.365275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5146c0-57b7-4277-b127-3af7697b0461 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.370463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.371402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.371680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.371946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] PATH_INFO: `/v3/projects/b53f565629784cf98c718c356e6dc28b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.373886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53f565629784cf98c718c356e6dc28b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.375350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.438009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53f565629784cf98c718c356e6dc28b', 'request_id': 'req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:07.445826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53f565629784cf98c718c356e6dc28b', 'request_id': 'req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:07.452628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53f565629784cf98c718c356e6dc28b', 'request_id': 'req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:07.470940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] The token cache is being invalidate because project b53f565629784cf98c718c356e6dc28b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:07.471280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:07.472168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4dd702cd-ef29-4890-a525-cb5ac08a5ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:07.473985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1013/2027] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:07 2022] DELETE /identity/v3/projects/b53f565629784cf98c718c356e6dc28b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:07.481609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.481971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.483319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.528007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:07.529955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:07.600317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1015/2028] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:07.608474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.608762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.609043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.629937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:07.631324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:07.639946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1014/2029] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:07.644302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.654855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabb0cf-e8f6-4a27-a8d0-e2f10861ce66 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.659885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.660560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.660838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.661107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.661844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:07.662147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.663789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.679247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1016/2030] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:07.681485 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.691243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe045e5-6f20-47ff-8203-825cd21af5be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.696511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.697116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.697378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.697675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.698372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.700355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.728483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1015/2031] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:07.732969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.743243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313cdebb-8678-4bb2-b4cd-4b70880952cb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.803361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.804068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.804347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.804612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.805345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.807319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.833517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1017/2032] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/users => generated 305 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:07.837802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.848635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2daec-6ee9-495c-9073-f0316d10e186 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.853675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.854411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.854800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.855111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.855838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.857443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.868340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1016/2033] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:07.872416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.882964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa8662-fcb6-4219-b75a-ca23d40ae3d0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.887829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da9d3c-55a9-4ecf-b6e2-79f88048b81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.888488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da9d3c-55a9-4ecf-b6e2-79f88048b81e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.888768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da9d3c-55a9-4ecf-b6e2-79f88048b81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.889045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da9d3c-55a9-4ecf-b6e2-79f88048b81e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.889827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da9d3c-55a9-4ecf-b6e2-79f88048b81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.891456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da9d3c-55a9-4ecf-b6e2-79f88048b81e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.892911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.904846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1018/2034] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:07.909286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:07.919703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609c93d-1b3e-4246-9bb7-22d0c73c215e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:07.924552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:07.925445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.925722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.925990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] PATH_INFO: `/v3/projects/c04bc0adc40d4966a6a2930add4c7ab2/users/df620ff35b784e7082788ecaef9871af/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:07.944964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04bc0adc40d4966a6a2930add4c7ab2, user_id=df620ff35b784e7082788ecaef9871af, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:07.947814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:07.950148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:07.967256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1017/2035] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:07 2022] PUT /identity/v3/projects/c04bc0adc40d4966a6a2930add4c7ab2/users/df620ff35b784e7082788ecaef9871af/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:07.977835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:07.978097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:07.978402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.006121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] MFA Rules not processed for user `df620ff35b784e7082788ecaef9871af`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:08.007385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:08.073232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1019/2036] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:07 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:08.078116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.142866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e153c0a-ecde-46af-a5ab-00bacbe7dbdf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.147966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.148638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.149239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.149538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.150225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.152176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.180825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1018/2037] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:08 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:08.185367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.196081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903780b7-7c66-4e95-9731-3f0a38a9099f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.254457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.255070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.255345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.255614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.256311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.258278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.280797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1020/2038] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:08 2022] POST /identity/v3/users => generated 311 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:08.284892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.296082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea6982-0104-407b-9a4c-a7c5852369d5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.301779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.302469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.302775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.303044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.303777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.305415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.317363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1019/2039] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:08.321797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.333030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8227-772f-46d2-b300-33ef28c67055 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.338444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.339398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.339686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.339963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] PATH_INFO: `/v3/projects/a4b537d1e4524a1088ead18116c737d3/users/a9de360e4d4743928d14577349c28b3c/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.358576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b537d1e4524a1088ead18116c737d3, user_id=a9de360e4d4743928d14577349c28b3c, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.361378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.377085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1021/2040] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:08 2022] PUT /identity/v3/projects/a4b537d1e4524a1088ead18116c737d3/users/a9de360e4d4743928d14577349c28b3c/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:08.381402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.392221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708186cf-09af-4c89-8d22-1baf7f1fb3a3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.458236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e55b43-dff7-4939-8534-285b6555ba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.458948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e55b43-dff7-4939-8534-285b6555ba77 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.459396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e55b43-dff7-4939-8534-285b6555ba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.459508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e55b43-dff7-4939-8534-285b6555ba77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.460206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e55b43-dff7-4939-8534-285b6555ba77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.461976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e55b43-dff7-4939-8534-285b6555ba77 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.473598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1020/2041] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:08.478268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.489571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad599d-f290-4bbf-a737-036e24e70fc6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.494872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.495646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.495925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.496194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] PATH_INFO: `/v3/projects/a4b537d1e4524a1088ead18116c737d3/users/a9de360e4d4743928d14577349c28b3c/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.498261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b537d1e4524a1088ead18116c737d3, user_id=a9de360e4d4743928d14577349c28b3c, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.501106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.512905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1022/2042] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:08 2022] PUT /identity/v3/projects/a4b537d1e4524a1088ead18116c737d3/users/a9de360e4d4743928d14577349c28b3c/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:08.525036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.525335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.525579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.554442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] MFA Rules not processed for user `a9de360e4d4743928d14577349c28b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:08.555675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:08.615770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.637942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1021/2043] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:08 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:08.649540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.651656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:08.698427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:08.715889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c93233-b26c-48fd-bfcc-3520fc102dde admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.720029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.720689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.720968 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.721307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:08.731195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:08.732642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:08.743030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1023/2044] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:08.929829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:08.939742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42d078-083f-45e1-a844-61ad854e1515 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:08.943800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a04e69-0d5d-4614-ae80-48edde392681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:08.944445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04e69-0d5d-4614-ae80-48edde392681 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:08.944724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04e69-0d5d-4614-ae80-48edde392681 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:08.944995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a04e69-0d5d-4614-ae80-48edde392681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:09.007232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a04e69-0d5d-4614-ae80-48edde392681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:09.008735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a04e69-0d5d-4614-ae80-48edde392681 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:09.018246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1022/2045] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:18.738732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:19.014044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:26.925322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.021982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf699847-6f60-49c4-8de5-4e631c65caf6 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.031446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.032582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.032919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.033199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] PATH_INFO: `/v3/users/df620ff35b784e7082788ecaef9871af` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.034580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df620ff35b784e7082788ecaef9871af)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.036679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.104964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df620ff35b784e7082788ecaef9871af'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.105932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df620ff35b784e7082788ecaef9871af'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.119436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df620ff35b784e7082788ecaef9871af'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.120336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df620ff35b784e7082788ecaef9871af'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.135867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df620ff35b784e7082788ecaef9871af'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.152374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1024/2046] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:26 2022] DELETE /identity/v3/users/df620ff35b784e7082788ecaef9871af => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:27.163257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.164016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.164016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.187989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:27.189915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:27.293960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1023/2047] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:27.305939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.308049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:27.418711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c942d5-47e2-41d7-bdd1-de54d1104d03 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.424941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.425716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.426117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.426527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.437601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.439022 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.456312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1025/2048] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:27.833727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:27.844337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:27.873746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757662f-c8a5-41e3-98b6-7994cb047a96 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:27.881229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:27.881926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:27.882090 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:27.882319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] PATH_INFO: `/v3/projects/c04bc0adc40d4966a6a2930add4c7ab2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:27.884128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04bc0adc40d4966a6a2930add4c7ab2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:27.886104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:27.950356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04bc0adc40d4966a6a2930add4c7ab2', 'request_id': 'req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.958645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04bc0adc40d4966a6a2930add4c7ab2', 'request_id': 'req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.965608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04bc0adc40d4966a6a2930add4c7ab2', 'request_id': 'req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.982282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] The token cache is being invalidate because project c04bc0adc40d4966a6a2930add4c7ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:27.983392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.983844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:27.985459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1024/2049] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:27 2022] DELETE /identity/v3/projects/c04bc0adc40d4966a6a2930add4c7ab2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:27.989902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:28.001415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8289de5-2f60-4c12-b504-b77caa02e417 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.024287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:28.071640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.072549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.072834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.073095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] PATH_INFO: `/v3/users/a9de360e4d4743928d14577349c28b3c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.074299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9de360e4d4743928d14577349c28b3c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.076379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.140768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9de360e4d4743928d14577349c28b3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:28.141452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9de360e4d4743928d14577349c28b3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:28.149320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9de360e4d4743928d14577349c28b3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:28.150009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9de360e4d4743928d14577349c28b3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:28.165460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b17cf3d8-2238-48cb-a402-235c962b1513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9de360e4d4743928d14577349c28b3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:28.180682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1026/2050] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:27 2022] DELETE /identity/v3/users/a9de360e4d4743928d14577349c28b3c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:28.752464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:28.754149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:28.772573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d6621-6bbc-41c8-b7db-d393748cb9ab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:28.854940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:28.856932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:28.857829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:28.858510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] PATH_INFO: `/v3/projects/a4b537d1e4524a1088ead18116c737d3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:28.861215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b537d1e4524a1088ead18116c737d3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:28.865266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:28.940810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:28.975546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b537d1e4524a1088ead18116c737d3', 'request_id': 'req-43c373d7-109b-40c3-85ff-444088dd44df'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:28.990936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b537d1e4524a1088ead18116c737d3', 'request_id': 'req-43c373d7-109b-40c3-85ff-444088dd44df'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:29.015756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b537d1e4524a1088ead18116c737d3', 'request_id': 'req-43c373d7-109b-40c3-85ff-444088dd44df'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:29.053392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] The token cache is being invalidate because project a4b537d1e4524a1088ead18116c737d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:29.053673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:29.055456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43c373d7-109b-40c3-85ff-444088dd44df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:29.057749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1025/2051] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:28 2022] DELETE /identity/v3/projects/a4b537d1e4524a1088ead18116c737d3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:29.065013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.065827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.066538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.104580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:29.105829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:29.181719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1027/2052] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:29.192102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.192460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.192921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.215475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:29.216788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:29.229399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1026/2053] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:29.234834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.251953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf049d5b-64da-4091-948d-bd8fdfe07ab9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.256989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.258026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.258500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.259039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.260362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:29.260898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.264375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.281732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1028/2054] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:29.285857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.300607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c46d8-7464-414f-8676-132e56db7c45 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.308770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.310162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.310678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.311213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.312617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.316749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.350544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1027/2055] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:29.354956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.368440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049b33e-cc4a-47ff-8ec7-2e7f272919d2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.394203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:29.455707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.457267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.457592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.458035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.458862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.461288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.490558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1029/2056] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/users => generated 283 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:29.491989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.505101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63307e1d-f971-4a6f-a192-9e715ebf24d0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.511082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.512183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.512779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.513296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.514629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.516727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.528681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1028/2057] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:29.533626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.544689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55317f26-4782-41d2-9aaf-147df3c29ac4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.551223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262dbda-e345-43f2-9a6b-482dd589628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.551890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262dbda-e345-43f2-9a6b-482dd589628b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.552182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262dbda-e345-43f2-9a6b-482dd589628b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.552470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262dbda-e345-43f2-9a6b-482dd589628b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.553172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262dbda-e345-43f2-9a6b-482dd589628b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.554801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262dbda-e345-43f2-9a6b-482dd589628b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.569588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1030/2058] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:29.576132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.591781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc1be2-a3e1-442d-9c87-57208fb3b0de admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.608811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ef9cb-8381-4910-8708-ded261488a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.610060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef9cb-8381-4910-8708-ded261488a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.610604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef9cb-8381-4910-8708-ded261488a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.611108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ef9cb-8381-4910-8708-ded261488a21 admin admin] PATH_INFO: `/v3/projects/41ff9488c32242a582744bdec6a517ef/users/683a53b5eb11446fb86814da55250db8/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.636740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef9cb-8381-4910-8708-ded261488a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ff9488c32242a582744bdec6a517ef, user_id=683a53b5eb11446fb86814da55250db8, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.650457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ef9cb-8381-4910-8708-ded261488a21 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.661014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1029/2059] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:29 2022] PUT /identity/v3/projects/41ff9488c32242a582744bdec6a517ef/users/683a53b5eb11446fb86814da55250db8/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:29.672207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.672520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.672797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.674326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.674895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.675691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.698744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] MFA Rules not processed for user `870b2f691e0145a6a28e541ad506af02`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:29.699983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:29.713334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] MFA Rules not processed for user `683a53b5eb11446fb86814da55250db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:29.714584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:29.828815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1030/2060] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:42:29.841634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1031/2061] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:29.845888 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:29.934453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f762ae-c6e0-47ff-9c36-9c8e2d2002d9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:29.935300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:29.941012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:29.941733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:29.942019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:29.942304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:29.943066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:29.945053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:29.988673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1031/2062] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:29.994668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.006658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f51059-d8b8-406e-a114-cb59d7aee651 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.081297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.082328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.082592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.083058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.084570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.087476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.113456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1032/2063] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:29 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:30.119147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.124280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.127241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.136938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc9f38-8baf-440f-9572-a90f1548d8f9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.148611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.148611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.148611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.150838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.150838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.151360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.160663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a202eec-1170-4882-b26c-5b395766e32e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.170922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1032/2064] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:30.173284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.173894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.175371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.175862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.176302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] PATH_INFO: `/v3/users/33badbd917a74b65b9eaf61dc1fefa3d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.178447 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33badbd917a74b65b9eaf61dc1fefa3d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.181953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.191648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffa420-ee5b-40e6-ada9-d027ea60ec92 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.199708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:30.200311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.202058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.202681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.203349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] PATH_INFO: `/v3/projects/505468401a5f4cf2b988ba66147ea92e/users/cbcb69f706c34977b0440d1b9314ae4d/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.227120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=505468401a5f4cf2b988ba66147ea92e, user_id=cbcb69f706c34977b0440d1b9314ae4d, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.230585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.252361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1033/2065] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:30 2022] PUT /identity/v3/projects/505468401a5f4cf2b988ba66147ea92e/users/cbcb69f706c34977b0440d1b9314ae4d/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:30.257216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.262472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33badbd917a74b65b9eaf61dc1fefa3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:30.263770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33badbd917a74b65b9eaf61dc1fefa3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:30.274941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33badbd917a74b65b9eaf61dc1fefa3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:30.275771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33badbd917a74b65b9eaf61dc1fefa3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:30.296737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc0795fd-4412-4001-8294-b07066a97127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33badbd917a74b65b9eaf61dc1fefa3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:30.318983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1033/2066] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:30 2022] DELETE /identity/v3/users/33badbd917a74b65b9eaf61dc1fefa3d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:30.323753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.324045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.324362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.349439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:30.352139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.359811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eae3f3-3c66-497c-aa37-506487d016fe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.440363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1034/2067] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:30.451603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.453250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.476581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.477487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.477975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.479501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.480844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.483964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.504115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1034/2068] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:42:30.509121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.511988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.531798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4475d-013c-4bf1-9113-2e331697e714 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.544397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ddbfd7-63db-4559-8c90-e046e6a1f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.546053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddbfd7-63db-4559-8c90-e046e6a1f82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.546536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddbfd7-63db-4559-8c90-e046e6a1f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.547021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ddbfd7-63db-4559-8c90-e046e6a1f82f admin admin] PATH_INFO: `/v3/projects/505468401a5f4cf2b988ba66147ea92e/users/cbcb69f706c34977b0440d1b9314ae4d/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.548366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a1dab-1eb5-4e42-a4ad-c53cb21e6af7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:30.550252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddbfd7-63db-4559-8c90-e046e6a1f82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=505468401a5f4cf2b988ba66147ea92e, user_id=cbcb69f706c34977b0440d1b9314ae4d, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.553066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ddbfd7-63db-4559-8c90-e046e6a1f82f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.553359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:30.554450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.555156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.556162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.578879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1035/2069] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:30 2022] PUT /identity/v3/projects/505468401a5f4cf2b988ba66147ea92e/users/cbcb69f706c34977b0440d1b9314ae4d/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:30.579068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:30.579068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:30.584814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:30.585192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:30.585395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:30.598547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1035/2070] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:30.627646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] MFA Rules not processed for user `cbcb69f706c34977b0440d1b9314ae4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:30.629318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.679924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:30.770847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1036/2071] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:42:30.783179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:30.784407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:30.857137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:30.960285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b417308-894c-428d-b313-51b10952c46e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.056728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.057563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.057983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.058407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.071381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.073259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.074157 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.077365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:31.086457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1036/2072] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:31.237638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206a040-7da3-499d-9352-fd5755aadd86 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.253613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.254050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.254336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.254609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] PATH_INFO: `/v3/projects/b7e29703c6204f09a7f14bb7984fa126` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.256079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7e29703c6204f09a7f14bb7984fa126)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.259365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.267129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.295442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7075f-202a-479e-877c-460a0ae8798f service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.318059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.320317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.320976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.321463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.327853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:31.347306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7e29703c6204f09a7f14bb7984fa126', 'request_id': 'req-aaba3535-d575-49a7-a0bb-cbde719ad0a2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.370940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7e29703c6204f09a7f14bb7984fa126', 'request_id': 'req-aaba3535-d575-49a7-a0bb-cbde719ad0a2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.380378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7e29703c6204f09a7f14bb7984fa126', 'request_id': 'req-aaba3535-d575-49a7-a0bb-cbde719ad0a2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.398617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] The token cache is being invalidate because project b7e29703c6204f09a7f14bb7984fa126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:31.399020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.402349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.406831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1037/2073] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:31 2022] DELETE /identity/v3/projects/b7e29703c6204f09a7f14bb7984fa126 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:31.409604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:31.411242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:31.492234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:31.496097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.499339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.518858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1037/2074] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:31.523125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba3535-d575-49a7-a0bb-cbde719ad0a2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:31.535067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:31.538172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:31.539195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:31.540124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] PATH_INFO: `/v3/users/7e4e24dd58464c1cb0df6d8fd3a6d5b6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:31.542685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e4e24dd58464c1cb0df6d8fd3a6d5b6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:31.547190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:31.648806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e4e24dd58464c1cb0df6d8fd3a6d5b6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.649825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e4e24dd58464c1cb0df6d8fd3a6d5b6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.651248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e4e24dd58464c1cb0df6d8fd3a6d5b6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.690578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e4e24dd58464c1cb0df6d8fd3a6d5b6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.703717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-68f6cb75-2cd1-4208-8e17-d65b4eb507c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e4e24dd58464c1cb0df6d8fd3a6d5b6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:31.723844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1038/2075] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:31 2022] DELETE /identity/v3/users/7e4e24dd58464c1cb0df6d8fd3a6d5b6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:32.287663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.307992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf93e28-4f78-41bc-baaa-3119b961eb92 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.407688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.409220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.409761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.410371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] PATH_INFO: `/v3/projects/36f0f238347f4f28a7f821acb2a9fd16` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.412524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f0f238347f4f28a7f821acb2a9fd16)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.419327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.441780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:32.497477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f0f238347f4f28a7f821acb2a9fd16', 'request_id': 'req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:32.507757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f0f238347f4f28a7f821acb2a9fd16', 'request_id': 'req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:32.521015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f0f238347f4f28a7f821acb2a9fd16', 'request_id': 'req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:32.543742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] The token cache is being invalidate because project 36f0f238347f4f28a7f821acb2a9fd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:32.544242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:32.545400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-983a9157-b3ce-4be7-aa8f-bebb6a6003ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:32.548378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1038/2076] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:32 2022] DELETE /identity/v3/projects/36f0f238347f4f28a7f821acb2a9fd16 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:32.557887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.559498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.560005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.593120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:32.595209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:32.709059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1039/2077] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:32.720710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.721037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.721347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.752961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:32.755072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:32.767509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1039/2078] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:32.774447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.798780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a63f6-3efe-4d89-a0f0-75b050535d9b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.804441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.804928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.805401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.805868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.807221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:32.807765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.810932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.836032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1040/2079] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:32.837853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.854713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83b971-399b-4e37-845d-511822eecea9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:32.861178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:32.862444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:32.862444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:32.862444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:32.863065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:32.864833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:32.920987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1040/2080] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:32 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:32.927004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:32.945546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00113bde-c0cb-40b2-97e8-0d6125669b2f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.020482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:33.030722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.031547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.032602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.033167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.034313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.038723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.070214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1041/2081] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:32 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:33.074141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.085061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55f123-74b5-45b0-a85f-005bfcd94221 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.091090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.091789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.092075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.092349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.093411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.095045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.112223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1041/2082] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:42:33.114246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.133411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2f6be-ab96-405c-9aa5-46242449e9bc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.141847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe42656-ea67-43a9-b453-8ed3e8a44f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.143132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42656-ea67-43a9-b453-8ed3e8a44f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.143576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42656-ea67-43a9-b453-8ed3e8a44f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.144003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe42656-ea67-43a9-b453-8ed3e8a44f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.145288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe42656-ea67-43a9-b453-8ed3e8a44f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.148807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe42656-ea67-43a9-b453-8ed3e8a44f2a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.167433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1042/2083] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:33.180727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.195377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0dcd9-9762-48ae-acb9-88be419f8cb5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.202651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.203024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.203569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.204068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] PATH_INFO: `/v3/projects/277b916dee814679bccd8101cb5af622/users/873db8c922eb470295a0ebac8734e360/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.225969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277b916dee814679bccd8101cb5af622, user_id=873db8c922eb470295a0ebac8734e360, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.229519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.257033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1042/2084] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:33 2022] PUT /identity/v3/projects/277b916dee814679bccd8101cb5af622/users/873db8c922eb470295a0ebac8734e360/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:33.270628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.270961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.271243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.313763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] MFA Rules not processed for user `873db8c922eb470295a0ebac8734e360`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:33.316513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:33.433299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1043/2085] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:33.438378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.570593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65b0a3-d1c3-47dc-b301-2115b5f234c7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.577885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.578934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.579498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.579804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.580759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.583798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.634813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1043/2086] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:33 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:33.637499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.652082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c315e-ea85-424a-b0c6-8929bc61d852 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.731911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.732747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.733188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.733585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.734415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.736670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.768187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1044/2087] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:33 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:33.774512 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.789268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd60ac-303c-465d-afc3-ec72a96ce347 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.808143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.809336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.809829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.810183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.811689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.815091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.830970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1044/2088] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:33.833360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.851213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e85848-5d93-4777-9404-aa45b88f60cd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.859468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.861082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.861567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.862045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] PATH_INFO: `/v3/projects/465fd23923c34ba0be9ab0aeaac73bcc/users/ac87f9288cec4176a61598a90d57e744/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.883113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=465fd23923c34ba0be9ab0aeaac73bcc, user_id=ac87f9288cec4176a61598a90d57e744, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.886341 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:33.905147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1045/2089] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:33 2022] PUT /identity/v3/projects/465fd23923c34ba0be9ab0aeaac73bcc/users/ac87f9288cec4176a61598a90d57e744/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:33.909531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:33.920235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e744b0-f0be-4063-b29c-e7e21f5fe03f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:33.929581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:33.994990 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be347cd4-1cdf-4acd-9d0c-175016c4aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:33.995572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be347cd4-1cdf-4acd-9d0c-175016c4aa70 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:33.995852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be347cd4-1cdf-4acd-9d0c-175016c4aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:33.996129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be347cd4-1cdf-4acd-9d0c-175016c4aa70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:33.996876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be347cd4-1cdf-4acd-9d0c-175016c4aa70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:33.999086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be347cd4-1cdf-4acd-9d0c-175016c4aa70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.012023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1045/2090] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:34.017204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:34.041389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c773863-8c51-416f-9d96-62213efc4c3f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:34.050512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:34.052971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.054070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.054683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] PATH_INFO: `/v3/projects/465fd23923c34ba0be9ab0aeaac73bcc/users/ac87f9288cec4176a61598a90d57e744/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.059614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=465fd23923c34ba0be9ab0aeaac73bcc, user_id=ac87f9288cec4176a61598a90d57e744, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:34.067150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.101725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1046/2091] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:34 2022] PUT /identity/v3/projects/465fd23923c34ba0be9ab0aeaac73bcc/users/ac87f9288cec4176a61598a90d57e744/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:34.115476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.116229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.116830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.172008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] MFA Rules not processed for user `ac87f9288cec4176a61598a90d57e744`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:34.174058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:34.271492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1046/2092] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:34.282639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:34.285314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:34.308353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:34.379799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7621d-d2b5-4fb4-829e-bd0121861ead admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:34.388939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:34.390453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.391321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.391707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.411488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:34.414366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.438014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1047/2093] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:34.631503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:34.648799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e38e5e-0e08-4734-9f3b-a9cd2b4c8673 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:34.655563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210224a0-ee4b-49ec-9c0b-c06322aaff4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:34.656653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210224a0-ee4b-49ec-9c0b-c06322aaff4f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:34.657129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210224a0-ee4b-49ec-9c0b-c06322aaff4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:34.657593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210224a0-ee4b-49ec-9c0b-c06322aaff4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:34.773222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210224a0-ee4b-49ec-9c0b-c06322aaff4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:34.774769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210224a0-ee4b-49ec-9c0b-c06322aaff4f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:34.794374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1047/2094] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:37.593969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:37.595641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:37.689533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00509-a88b-4b7a-a4c9-1e509529ff6a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:37.697510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:37.699110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:37.699587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:37.700088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] PATH_INFO: `/v3/users/683a53b5eb11446fb86814da55250db8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:37.702280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=683a53b5eb11446fb86814da55250db8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:37.706143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:37.811523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683a53b5eb11446fb86814da55250db8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:37.812667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683a53b5eb11446fb86814da55250db8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:37.813932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683a53b5eb11446fb86814da55250db8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:37.839654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683a53b5eb11446fb86814da55250db8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:37.857386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683a53b5eb11446fb86814da55250db8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:37.890132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1048/2095] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:37 2022] DELETE /identity/v3/users/683a53b5eb11446fb86814da55250db8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:37.902136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:37.902136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:37.902136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:37.910706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:37.914290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:37.932212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:37.934349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:37.984908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:38.033071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:38.047562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1048/2096] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:38.079773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:38.082127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:38.094340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669999fc-02b4-4701-980a-1c632c04b8b8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.104388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.105974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.106228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.106735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] PATH_INFO: `/v3/users/873db8c922eb470295a0ebac8734e360` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.108451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=873db8c922eb470295a0ebac8734e360)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.112964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:38.210292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab5897-6a12-42ee-9b33-f8b503de074d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.213727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873db8c922eb470295a0ebac8734e360'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:38.214657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873db8c922eb470295a0ebac8734e360'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:38.219809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873db8c922eb470295a0ebac8734e360'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:38.221446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.222875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.223343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.223943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.227146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:38.239713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873db8c922eb470295a0ebac8734e360'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:38.249803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873db8c922eb470295a0ebac8734e360'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:38.268829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.271809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c1614-ede6-4c27-b2b4-52b29928f6bc service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:38.291793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1049/2097] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:42:38.292140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1049/2098] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:37 2022] DELETE /identity/v3/users/873db8c922eb470295a0ebac8734e360 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:38.303407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.305341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.305868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.333415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:38.335529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:38.481251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1050/2099] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:38.498564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:38.500489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:38.613661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba725dc0-4ce6-4587-a6bb-f09772c2de74 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.620062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.621218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.621716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.622180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.640439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.643691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:38.663316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1050/2100] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:42:38.835576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:38.837217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:38.854226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b8612-da5d-4ae8-804e-6986795ebd90 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:38.864286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:38.875727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:38.876654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:38.878318 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] PATH_INFO: `/v3/projects/41ff9488c32242a582744bdec6a517ef` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:38.884470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ff9488c32242a582744bdec6a517ef)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:38.903375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.008708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ff9488c32242a582744bdec6a517ef', 'request_id': 'req-d2035237-772a-481e-a32e-af328a3bad3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.029110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ff9488c32242a582744bdec6a517ef', 'request_id': 'req-d2035237-772a-481e-a32e-af328a3bad3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.043617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ff9488c32242a582744bdec6a517ef', 'request_id': 'req-d2035237-772a-481e-a32e-af328a3bad3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.059588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:39.069379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] The token cache is being invalidate because project 41ff9488c32242a582744bdec6a517ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:39.070389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.073267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.076035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1051/2101] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:38 2022] DELETE /identity/v3/projects/41ff9488c32242a582744bdec6a517ef => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:39.081663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.101050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f979da-76e8-4e84-be44-acad37ceb719 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.200384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.202273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.202739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.203295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] PATH_INFO: `/v3/users/cbcb69f706c34977b0440d1b9314ae4d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.205360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcb69f706c34977b0440d1b9314ae4d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.208936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.215058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:39.217410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.219621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:39.252275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2035237-772a-481e-a32e-af328a3bad3d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.257983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.259118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.259491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.259857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] PATH_INFO: `/v3/projects/277b916dee814679bccd8101cb5af622` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.261373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277b916dee814679bccd8101cb5af622)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.263867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.328646 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcb69f706c34977b0440d1b9314ae4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.330014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcb69f706c34977b0440d1b9314ae4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.331173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcb69f706c34977b0440d1b9314ae4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.352528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcb69f706c34977b0440d1b9314ae4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.357029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277b916dee814679bccd8101cb5af622', 'request_id': 'req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.369009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcb69f706c34977b0440d1b9314ae4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.369378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277b916dee814679bccd8101cb5af622', 'request_id': 'req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.397232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1051/2102] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:39 2022] DELETE /identity/v3/users/cbcb69f706c34977b0440d1b9314ae4d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:39.400392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277b916dee814679bccd8101cb5af622', 'request_id': 'req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.432552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] The token cache is being invalidate because project 277b916dee814679bccd8101cb5af622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:39.433457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.435394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.442857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1052/2103] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:39 2022] DELETE /identity/v3/projects/277b916dee814679bccd8101cb5af622 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:39.445317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:39.447063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:39.564605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f506a07-9a2b-4613-9f81-3ecb9039d02e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:39.574937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:39.576706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:39.577319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:39.577934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] PATH_INFO: `/v3/users/ac87f9288cec4176a61598a90d57e744` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:39.580404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac87f9288cec4176a61598a90d57e744)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:39.584573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:39.674059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac87f9288cec4176a61598a90d57e744'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.676008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac87f9288cec4176a61598a90d57e744'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.676795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac87f9288cec4176a61598a90d57e744'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.701128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac87f9288cec4176a61598a90d57e744'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.712568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-56a4c4a2-9111-48de-8c2e-ead4562064ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac87f9288cec4176a61598a90d57e744'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:39.726621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:39.734854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1052/2104] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:42:39 2022] DELETE /identity/v3/users/ac87f9288cec4176a61598a90d57e744 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:40.008175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:40.011693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.151636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac29ebf-fb4d-436e-8bbf-b39d171817d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:40.239295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:40.243191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.243191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.243191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] PATH_INFO: `/v3/projects/505468401a5f4cf2b988ba66147ea92e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.243191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] RBAC: Authorizing `identity:delete_project(project_id=505468401a5f4cf2b988ba66147ea92e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:40.244126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:40.307247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:40.333520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505468401a5f4cf2b988ba66147ea92e', 'request_id': 'req-93ccb673-1da3-438a-a393-33e4f970257a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.342925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505468401a5f4cf2b988ba66147ea92e', 'request_id': 'req-93ccb673-1da3-438a-a393-33e4f970257a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.349846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505468401a5f4cf2b988ba66147ea92e', 'request_id': 'req-93ccb673-1da3-438a-a393-33e4f970257a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.364911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] The token cache is being invalidate because project 505468401a5f4cf2b988ba66147ea92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:40.365215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.365840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.367473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1053/2105] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:40 2022] DELETE /identity/v3/projects/505468401a5f4cf2b988ba66147ea92e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:40.372890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00146b-dc34-41ad-a383-34008a753ac9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.373246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00146b-dc34-41ad-a383-34008a753ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.373612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00146b-dc34-41ad-a383-34008a753ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.411974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fe00146b-dc34-41ad-a383-34008a753ac9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:40.414468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fe00146b-dc34-41ad-a383-34008a753ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.436361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:40.438638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.524543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1053/2106] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:40.532959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.533223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.533526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.561895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:40.563898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.566517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccb673-1da3-438a-a393-33e4f970257a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:40.577237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:40.578211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1054/2107] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:42:40.578396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.578538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.578803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] PATH_INFO: `/v3/projects/465fd23923c34ba0be9ab0aeaac73bcc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.580143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465fd23923c34ba0be9ab0aeaac73bcc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:40.582121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:40.584608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:40.600798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628810e-8c07-460b-9772-6dc448b8e6c1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:40.611290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:40.612462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.613326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.613447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.614634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:40.615245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:40.618343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:40.643796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1055/2108] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:42:40.649915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:40.663575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68db89b-ba5d-4532-b737-24618ec0688d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:40.665704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465fd23923c34ba0be9ab0aeaac73bcc', 'request_id': 'req-7660b8d7-3017-4d53-9ae2-309e46aed7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.677217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465fd23923c34ba0be9ab0aeaac73bcc', 'request_id': 'req-7660b8d7-3017-4d53-9ae2-309e46aed7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.688240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465fd23923c34ba0be9ab0aeaac73bcc', 'request_id': 'req-7660b8d7-3017-4d53-9ae2-309e46aed7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.712278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] The token cache is being invalidate because project 465fd23923c34ba0be9ab0aeaac73bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:42:40.712934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.713637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7660b8d7-3017-4d53-9ae2-309e46aed7e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:42:40.720047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1054/2109] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:42:40 2022] DELETE /identity/v3/projects/465fd23923c34ba0be9ab0aeaac73bcc => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:40.722683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c257aa8-be58-4cf1-b015-a2975c357421 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.723273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c257aa8-be58-4cf1-b015-a2975c357421 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.723814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c257aa8-be58-4cf1-b015-a2975c357421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.748347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c257aa8-be58-4cf1-b015-a2975c357421 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:40.749619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c257aa8-be58-4cf1-b015-a2975c357421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.754493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:40.755326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.756232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.756232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.756477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:40.760132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:40.761286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1055/2110] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:40.773272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:40.773272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:40.773272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:40.808572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:40.810683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.822356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1056/2111] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:40 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:42:40.823045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1056/2112] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:40.828330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:40.830170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:40.831921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:40.848964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f25d3-27ad-4e78-9979-238861ad0d6c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:40.918997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:40.988305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f3088-91cc-4b7f-838b-50640f449194 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:40.998871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.000949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.001585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.002215 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.003700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.008663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.022610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.025267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.025778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.026270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.027121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:41.028922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:42:41.030001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.033335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.057220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1057/2113] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:42:41.062916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1057/2114] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:40 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:41.063216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.064138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.077579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea358b-caf5-413d-8d57-b8564bad017e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.083953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98633ff6-3505-48b1-b16b-800a2e3ba06b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.089765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.090425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.091522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.091934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.092335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.093166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.095661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.095846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.097839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.099076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.101108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.106128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.145331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1058/2115] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:41.150933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.151464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1058/2116] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:42:41 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:41.160611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.179149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7422352-3f37-4204-9ca8-d9a15e0df7f6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.311546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639c44b-0f87-42d9-9052-3157a6a563d0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.315285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.316310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.316749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.317181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.317605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.318442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.318606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.319036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.319616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.320279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.322279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.322464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.342859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1059/2117] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:41.348948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.360970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1059/2118] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:42:41 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:42:41.365211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.365605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b632ad43-8cf8-40c0-8301-19bc8cd68845 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.373140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbf808-44ad-4da9-8f40-0c09f4a1fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.374595 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbf808-44ad-4da9-8f40-0c09f4a1fe00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.375049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbf808-44ad-4da9-8f40-0c09f4a1fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.375715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbf808-44ad-4da9-8f40-0c09f4a1fe00 admin admin] PATH_INFO: `/v3/projects/0b11a7f2b3534b65b08e7f1776d9d046/users/ede21ce73d0645adad6c625a88f1359f/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.379575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecf05c-7041-4cc8-ac74-78451a603f8a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.389229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.390713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.391399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.392321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.394111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.398188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.403315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbf808-44ad-4da9-8f40-0c09f4a1fe00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b11a7f2b3534b65b08e7f1776d9d046, user_id=ede21ce73d0645adad6c625a88f1359f, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.408532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbf808-44ad-4da9-8f40-0c09f4a1fe00 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.418423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1060/2119] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:42:41.424023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.428477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1060/2120] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:41 2022] PUT /identity/v3/projects/0b11a7f2b3534b65b08e7f1776d9d046/users/ede21ce73d0645adad6c625a88f1359f/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:42:41.437162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81001fa-dfc4-49a5-b430-0c947161c493 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.440497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.440866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.441203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.479415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] MFA Rules not processed for user `ede21ce73d0645adad6c625a88f1359f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:41.481071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:41.533877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.535051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.535513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.536104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.537822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.540387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.559563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1061/2121] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:42:41.564793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.589274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89150-f135-4883-a5d0-546f678995b2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.596641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bbdef-a64c-4d10-96c6-f438a868779d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.597564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bbdef-a64c-4d10-96c6-f438a868779d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.597851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bbdef-a64c-4d10-96c6-f438a868779d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.598130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bbdef-a64c-4d10-96c6-f438a868779d admin admin] PATH_INFO: `/v3/projects/34e1c75d3f204befb221cbafb8fdae7c/users/c34a0e6c8c424f7b97fef8059c5f3c96/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.601689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1061/2122] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:42:41.617965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bbdef-a64c-4d10-96c6-f438a868779d admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e1c75d3f204befb221cbafb8fdae7c, user_id=c34a0e6c8c424f7b97fef8059c5f3c96, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.620823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bbdef-a64c-4d10-96c6-f438a868779d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.626286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.627748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:41.644543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1062/2123] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:42:41 2022] PUT /identity/v3/projects/34e1c75d3f204befb221cbafb8fdae7c/users/c34a0e6c8c424f7b97fef8059c5f3c96/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:42:41.655646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:41.661779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.662272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.662763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.697942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2933c5-d335-4550-9ddc-c26d29b23a1d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.700582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] MFA Rules not processed for user `c34a0e6c8c424f7b97fef8059c5f3c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:41.701994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:41.718199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:41.764365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abbf034-d3a6-4b3d-9f60-38c9aa501af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.765458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf034-d3a6-4b3d-9f60-38c9aa501af0 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.765934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf034-d3a6-4b3d-9f60-38c9aa501af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.766399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abbf034-d3a6-4b3d-9f60-38c9aa501af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.841513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1063/2124] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:42:41.860119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:42:41.861775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:41.877788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbf034-d3a6-4b3d-9f60-38c9aa501af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.878884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60342-187c-4d70-8655-1084ed07128c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:42:41.880781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abbf034-d3a6-4b3d-9f60-38c9aa501af0 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.885599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:42:41.886983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:41.887606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:41.888178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:41.905598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:42:41.909210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:42:41.914145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1062/2125] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:42:41.928141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1064/2126] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:42:43.484365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1480d-3d90-428e-aecb-69db175b0283 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:42:43.485179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1480d-3d90-428e-aecb-69db175b0283 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:42:43.485513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1480d-3d90-428e-aecb-69db175b0283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:42:43.515000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d2a1480d-3d90-428e-aecb-69db175b0283 None None] MFA Rules not processed for user `c34a0e6c8c424f7b97fef8059c5f3c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:42:43.517268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2a1480d-3d90-428e-aecb-69db175b0283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:42:43.530808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1063/2127] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:42:51.919693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=27 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:42:53.507825 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=62 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:09.508896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:09.659272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86b2d2-fba2-4951-9b88-955e1ce9ef2e service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:09.672309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:09.675412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:09.675908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:09.676628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] PATH_INFO: `/v3/users/ede21ce73d0645adad6c625a88f1359f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:09.680756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede21ce73d0645adad6c625a88f1359f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:09.692203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:09.779300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede21ce73d0645adad6c625a88f1359f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:09.782896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede21ce73d0645adad6c625a88f1359f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:09.784086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede21ce73d0645adad6c625a88f1359f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:09.805750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede21ce73d0645adad6c625a88f1359f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:09.815762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede21ce73d0645adad6c625a88f1359f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:09.833751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1065/2128] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:43:09 2022] DELETE /identity/v3/users/ede21ce73d0645adad6c625a88f1359f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:09.838631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:09.839296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:09.839406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:09.864397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:09.866283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:09.972869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1064/2129] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:09.982312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:09.984777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:10.092035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc9c8-2bc2-44e0-b5ba-3e0f612931c8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:10.098544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e613b4-a8e4-4cb9-b08a-ea28c1c9f16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:10.099701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e613b4-a8e4-4cb9-b08a-ea28c1c9f16f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:10.100214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e613b4-a8e4-4cb9-b08a-ea28c1c9f16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:10.100668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e613b4-a8e4-4cb9-b08a-ea28c1c9f16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:10.125866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e613b4-a8e4-4cb9-b08a-ea28c1c9f16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:10.128276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e613b4-a8e4-4cb9-b08a-ea28c1c9f16f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:10.148124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1066/2130] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:43:10.708661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:10.711051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:10.750543 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f4e74-a184-4a35-bbc6-4b827cea0228 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:10.756020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:10.757542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:10.758056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:10.758636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] PATH_INFO: `/v3/projects/0b11a7f2b3534b65b08e7f1776d9d046` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:10.760402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b11a7f2b3534b65b08e7f1776d9d046)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:10.762443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:10.841557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b11a7f2b3534b65b08e7f1776d9d046', 'request_id': 'req-f467841a-beb3-4059-8003-a96736c7b0e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:10.851007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b11a7f2b3534b65b08e7f1776d9d046', 'request_id': 'req-f467841a-beb3-4059-8003-a96736c7b0e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:10.859395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b11a7f2b3534b65b08e7f1776d9d046', 'request_id': 'req-f467841a-beb3-4059-8003-a96736c7b0e1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:10.878909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] The token cache is being invalidate because project 0b11a7f2b3534b65b08e7f1776d9d046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:43:10.879301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:10.880021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f467841a-beb3-4059-8003-a96736c7b0e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:10.881702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1065/2131] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:43:10 2022] DELETE /identity/v3/projects/0b11a7f2b3534b65b08e7f1776d9d046 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:10.886851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ca69a-737a-4218-a04b-2838460948aa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:10.887437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ca69a-737a-4218-a04b-2838460948aa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:10.887829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ca69a-737a-4218-a04b-2838460948aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:10.911663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-949ca69a-737a-4218-a04b-2838460948aa None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:10.913856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-949ca69a-737a-4218-a04b-2838460948aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:10.938346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:11.022328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1067/2132] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:11.033648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.034059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.034456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.060372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:11.061518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:11.071348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1066/2133] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:11.078253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ca69a-737a-4218-a04b-2838460948aa None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.091950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ca69a-737a-4218-a04b-2838460948aa None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.098321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.099079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.099348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.099782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.100982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:11.101292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.102953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.119576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1068/2134] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:11.126049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.138446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e27c26-bfd9-4eed-8c09-6825c41376f8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.150643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33468710-c6b1-4536-b3b8-a93c2098962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.153091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33468710-c6b1-4536-b3b8-a93c2098962a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.153091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33468710-c6b1-4536-b3b8-a93c2098962a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.153091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33468710-c6b1-4536-b3b8-a93c2098962a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.153091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33468710-c6b1-4536-b3b8-a93c2098962a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.154542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33468710-c6b1-4536-b3b8-a93c2098962a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.187470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1067/2135] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:11.192777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.204961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ab13f-9c26-434c-b083-265027522ed0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.260702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.260978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.261276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.281997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.282690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.283007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.283309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.283883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] MFA Rules not processed for user `73d50022c5f94444afc67ea810e4d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:11.284039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.285693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:11.285981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.315252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1069/2136] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:11.317097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.328649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b048c-6428-40c3-88f9-e718a5b02c3e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.340155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.341185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.342121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.342121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.342533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.344413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.358801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1070/2137] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:43:11.364042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.365296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:11.377080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1068/2138] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:11.381573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c897c-f5a1-4298-bdbb-e4e2bf79b55e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.387704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa0b55-8d5b-44d6-88c9-4ef857d4bbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.388735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0b55-8d5b-44d6-88c9-4ef857d4bbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.389208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0b55-8d5b-44d6-88c9-4ef857d4bbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.389662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa0b55-8d5b-44d6-88c9-4ef857d4bbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.391012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0b55-8d5b-44d6-88c9-4ef857d4bbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.394106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa0b55-8d5b-44d6-88c9-4ef857d4bbf6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.414290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1071/2139] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:43:11.420966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.436874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4e441-da41-4fa8-845d-5667ff40b9aa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.444493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.446064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.447344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.447344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] PATH_INFO: `/v3/projects/a1e384a39cd840578bd2f265841b8f6c/users/365d72182d8145898812cc6f2cc6b27e/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.471421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e384a39cd840578bd2f265841b8f6c, user_id=365d72182d8145898812cc6f2cc6b27e, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.475479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.500604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1069/2140] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:43:11 2022] PUT /identity/v3/projects/a1e384a39cd840578bd2f265841b8f6c/users/365d72182d8145898812cc6f2cc6b27e/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:11.517556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.518367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.518537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.559055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] MFA Rules not processed for user `365d72182d8145898812cc6f2cc6b27e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:11.560687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:11.669325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1072/2141] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:11.674835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.782392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cc39e-b376-49ca-ae7a-2f2ad46d5b48 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.798191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.800420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.800853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.801319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.802579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.806195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.854873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1070/2142] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:11.860912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:11.875060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5c026-c1d5-4109-8482-9b1ee4ff29ce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:11.880575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:11.953763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:11.956662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:11.956662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:11.956662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:11.956662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:11.958023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:11.986807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1073/2143] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:43:11 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:11.992399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.007522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b81499-9ed0-49b9-a162-362ca85c41c4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.012683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.013375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.013658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.013940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.014681 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.016346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.029336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1071/2144] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:43:12.034628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.046357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee6be-706e-4fc4-aaba-f0fea5c19166 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.051590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64649792-95b9-4f12-9c30-61a566ff80c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.052441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64649792-95b9-4f12-9c30-61a566ff80c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.052794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64649792-95b9-4f12-9c30-61a566ff80c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.053152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64649792-95b9-4f12-9c30-61a566ff80c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.055004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64649792-95b9-4f12-9c30-61a566ff80c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.056497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64649792-95b9-4f12-9c30-61a566ff80c8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.074940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1074/2145] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:43:12.082294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.096247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152dfef-6e6a-4d31-91d2-e592af1a17c0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.101253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.102162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.102438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.102706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] PATH_INFO: `/v3/projects/265e38319f8f4698a0df19926feab285/users/1d7e745aaa1d4a55933f1ac351c3a39c/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.122601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=265e38319f8f4698a0df19926feab285, user_id=1d7e745aaa1d4a55933f1ac351c3a39c, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.125620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.149956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1072/2146] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:43:12 2022] PUT /identity/v3/projects/265e38319f8f4698a0df19926feab285/users/1d7e745aaa1d4a55933f1ac351c3a39c/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:12.167598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.167970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.168314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.209828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] MFA Rules not processed for user `1d7e745aaa1d4a55933f1ac351c3a39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:12.211162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:12.317336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1075/2147] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:12.324287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.406090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6726364-256b-44b3-84ba-c0aa95921818 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.412662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.413335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.413614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.413886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.414599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.416601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.452721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1073/2148] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:43:12 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:43:12.458470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.485166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdd1d-825e-4660-8db9-6466bf0826d0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.572054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.573217 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.573479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.573789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.574494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.576643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.607655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1076/2149] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:43:12 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:12.611797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.623575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369716-fdc6-4cc8-b703-3d089940aef2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.628850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.629540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.629819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.630100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.630872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.632617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.639441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:12.648156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1074/2150] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:43:12.652418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.659659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:12.663915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781bb87-5ec8-4ff2-8c0a-e48838297665 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.671437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.672448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.672775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.673065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] PATH_INFO: `/v3/projects/7d7d5dc3b3b74751b9e03e107612201c/users/afad0c6e5d044bf99226a476854eea3f/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.694552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7d5dc3b3b74751b9e03e107612201c, user_id=afad0c6e5d044bf99226a476854eea3f, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.697820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.724000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1077/2151] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:43:12 2022] PUT /identity/v3/projects/7d7d5dc3b3b74751b9e03e107612201c/users/afad0c6e5d044bf99226a476854eea3f/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:12.730459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.744426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b57dd-65b5-4237-8c23-b4df81ed2ab5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.818278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae8820-70b3-47ba-94d0-b54a1237509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.818994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae8820-70b3-47ba-94d0-b54a1237509c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.819275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae8820-70b3-47ba-94d0-b54a1237509c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.819548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae8820-70b3-47ba-94d0-b54a1237509c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.820285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae8820-70b3-47ba-94d0-b54a1237509c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.821887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae8820-70b3-47ba-94d0-b54a1237509c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.833485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1075/2152] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 14:43:12.839613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:12.851441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55b075-33ff-47ab-80dd-a04f18797d8b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:12.856550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:12.857457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.857739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.858152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] PATH_INFO: `/v3/projects/7d7d5dc3b3b74751b9e03e107612201c/users/afad0c6e5d044bf99226a476854eea3f/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.860547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7d5dc3b3b74751b9e03e107612201c, user_id=afad0c6e5d044bf99226a476854eea3f, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:12.863359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:12.875511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1078/2153] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:43:12 2022] PUT /identity/v3/projects/7d7d5dc3b3b74751b9e03e107612201c/users/afad0c6e5d044bf99226a476854eea3f/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:43:12.892290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:12.893208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:12.893478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:12.926109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] MFA Rules not processed for user `afad0c6e5d044bf99226a476854eea3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:12.927365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:13.034607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1076/2154] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:13.046579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:13.053281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:13.094313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:13.095960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:13.161716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ce5b4-cb24-4877-bb7c-51b22878da5b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:13.167608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:13.168431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:13.169136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:13.170156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:13.265144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:13.270212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4139e4f-be62-4323-87c7-300b68e5e76b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:13.275153 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:13.277815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:13.279553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:13.280709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:13.281178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:13.281620 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:13.284947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:13.296326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1079/2155] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:43:13.305018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:13.308388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:13.329937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1077/2156] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:43:15.044325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:15.046009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:15.061562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27f45b-ed5b-4063-ab2d-01a40579e590 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:15.068042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbe4bd-6717-456d-975c-b933d9cae178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:15.069205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe4bd-6717-456d-975c-b933d9cae178 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:15.070044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe4bd-6717-456d-975c-b933d9cae178 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:15.070381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbe4bd-6717-456d-975c-b933d9cae178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:15.199770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbe4bd-6717-456d-975c-b933d9cae178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:15.201805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbe4bd-6717-456d-975c-b933d9cae178 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:15.219098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1080/2157] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:43:15.988173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:16.000349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eef75-fe36-477d-945b-e618bbf4e7da service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:16.014247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:16.015474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:16.015474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:16.015474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:16.031129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:16.032621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:16.046115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1078/2158] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:43:21.728902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:21.729535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:21.729535 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:21.765147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:21.842304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:21.844273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:21.988282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1081/2159] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:22.047689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:22.049201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:22.172255 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab61fb8-a220-49d2-bc71-a67f65cbec9c service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:22.177470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2f81f-de3f-4612-a9cb-9fd060d42628 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:22.178129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f81f-de3f-4612-a9cb-9fd060d42628 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:22.178408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f81f-de3f-4612-a9cb-9fd060d42628 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:22.178675 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2f81f-de3f-4612-a9cb-9fd060d42628 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:22.188515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2f81f-de3f-4612-a9cb-9fd060d42628 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:22.190160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2f81f-de3f-4612-a9cb-9fd060d42628 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:22.205072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1079/2160] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:43:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 14:43:24.832631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:24.988583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030edc2a-cc8e-40a3-8dcc-43db9e4a1e23 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:25.004763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:25.006581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.007393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.008006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] PATH_INFO: `/v3/users/365d72182d8145898812cc6f2cc6b27e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.010817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365d72182d8145898812cc6f2cc6b27e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:25.015602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:25.123247 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365d72182d8145898812cc6f2cc6b27e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:25.124364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365d72182d8145898812cc6f2cc6b27e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:25.125152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365d72182d8145898812cc6f2cc6b27e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:25.146204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365d72182d8145898812cc6f2cc6b27e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:25.161580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365d72182d8145898812cc6f2cc6b27e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:25.168002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:25.187410 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1082/2161] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:43:24 2022] DELETE /identity/v3/users/365d72182d8145898812cc6f2cc6b27e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:43:25.192801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.193614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.193614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.226090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:25.228413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:25.369448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1080/2162] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:25.382567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:25.388531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:25.569540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a195f8-3266-475e-9b29-8b151a5f46c1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:25.585369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:25.586419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:25.586706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:25.587059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:25.597649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:25.599204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:25.610851 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1083/2163] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:43:26.345008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:26.350889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:26.382322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb37e08-f1ed-46a0-b102-baef2ad1fd5e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:26.392049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:26.394706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:26.395835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:26.396410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] PATH_INFO: `/v3/projects/a1e384a39cd840578bd2f265841b8f6c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:26.398325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e384a39cd840578bd2f265841b8f6c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:26.402380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:26.489771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:26.498166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e384a39cd840578bd2f265841b8f6c', 'request_id': 'req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.516955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e384a39cd840578bd2f265841b8f6c', 'request_id': 'req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.535032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e384a39cd840578bd2f265841b8f6c', 'request_id': 'req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.569500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] The token cache is being invalidate because project a1e384a39cd840578bd2f265841b8f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:43:26.569878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.572522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.580350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1081/2164] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:43:26 2022] DELETE /identity/v3/projects/a1e384a39cd840578bd2f265841b8f6c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:26.582461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:26.599257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbd2dd-d196-4107-9cf7-50e7ecf52dee service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:26.706911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:26.707810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:26.708081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:26.708431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] PATH_INFO: `/v3/users/1d7e745aaa1d4a55933f1ac351c3a39c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:26.717466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d7e745aaa1d4a55933f1ac351c3a39c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:26.720202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:26.859119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d7e745aaa1d4a55933f1ac351c3a39c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.860310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d7e745aaa1d4a55933f1ac351c3a39c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.861106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d7e745aaa1d4a55933f1ac351c3a39c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.887978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d7e745aaa1d4a55933f1ac351c3a39c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.904179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d7e745aaa1d4a55933f1ac351c3a39c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:26.923394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1084/2165] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:43:26 2022] DELETE /identity/v3/users/1d7e745aaa1d4a55933f1ac351c3a39c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:27.832677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:27.835842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:27.876332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7af6f-8eee-4463-9d5e-6e8a9fc6bde6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:28.028143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:28.029036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:28.029312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:28.029579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] PATH_INFO: `/v3/projects/265e38319f8f4698a0df19926feab285` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:28.034656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265e38319f8f4698a0df19926feab285)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:28.037666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:28.130024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265e38319f8f4698a0df19926feab285', 'request_id': 'req-81de9fc8-630c-469d-92c8-eaea1bc22d92'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.145569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265e38319f8f4698a0df19926feab285', 'request_id': 'req-81de9fc8-630c-469d-92c8-eaea1bc22d92'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.153387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265e38319f8f4698a0df19926feab285', 'request_id': 'req-81de9fc8-630c-469d-92c8-eaea1bc22d92'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.171905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] The token cache is being invalidate because project 265e38319f8f4698a0df19926feab285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:43:28.172481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.173483 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.181397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1082/2166] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:43:27 2022] DELETE /identity/v3/projects/265e38319f8f4698a0df19926feab285 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:28.182703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:28.186814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:28.206788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d284d6-0d33-41a8-bf62-4e13c4fc8c58 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:28.233528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:28.339545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:28.341128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:28.341644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:28.342118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] PATH_INFO: `/v3/users/afad0c6e5d044bf99226a476854eea3f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:28.344809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=afad0c6e5d044bf99226a476854eea3f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:28.348726 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:28.439784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afad0c6e5d044bf99226a476854eea3f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.439997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afad0c6e5d044bf99226a476854eea3f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.440716 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afad0c6e5d044bf99226a476854eea3f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.486016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afad0c6e5d044bf99226a476854eea3f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.502424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b3ed5915-093d-4700-b11f-17b8c90667bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afad0c6e5d044bf99226a476854eea3f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:28.524769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1085/2167] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:43:28 2022] DELETE /identity/v3/users/afad0c6e5d044bf99226a476854eea3f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:29.391331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:29.394469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:29.421755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de9fc8-630c-469d-92c8-eaea1bc22d92 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:29.512531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:29.513655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:29.514043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:29.514423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] PATH_INFO: `/v3/projects/7d7d5dc3b3b74751b9e03e107612201c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:29.520594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7d5dc3b3b74751b9e03e107612201c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:29.524021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:29.528104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:29.621758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7d5dc3b3b74751b9e03e107612201c', 'request_id': 'req-91c9c178-a609-4d29-93fd-51f5c74eb1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:29.629974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7d5dc3b3b74751b9e03e107612201c', 'request_id': 'req-91c9c178-a609-4d29-93fd-51f5c74eb1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:29.638819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7d5dc3b3b74751b9e03e107612201c', 'request_id': 'req-91c9c178-a609-4d29-93fd-51f5c74eb1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:29.656441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] The token cache is being invalidate because project 7d7d5dc3b3b74751b9e03e107612201c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:43:29.656769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:29.657524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-91c9c178-a609-4d29-93fd-51f5c74eb1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:29.659571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1083/2168] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:43:29 2022] DELETE /identity/v3/projects/7d7d5dc3b3b74751b9e03e107612201c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:29.665735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:29.666830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:29.667273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:29.713130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:29.717059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:29.833839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1086/2169] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:29.845327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:29.845657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:29.846071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:29.876415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:29.877703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:29.901912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1084/2170] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:29.907033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:29.917730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8148ea-1301-461e-abf2-bac66a690d2e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:29.922651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:29.923362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:29.923641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:29.923914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:29.924939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:29.925256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:29.926950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:29.942203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1087/2171] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:43:29.948487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:29.958994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d2d6f-38f4-40bb-b6b1-81aa7eefa512 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:29.966969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:29.967906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:29.968185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:29.968508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:29.970083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:29.971415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.024137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1085/2172] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:43:29 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:43:30.030377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:30.053770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf668f42-e071-4424-8f9e-47f2199a2125 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:30.121644 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:30.179099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:30.180626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.181188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.181744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.183295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:30.186933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.230691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1088/2173] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:43:30 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:30.236036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:30.255499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426178a5-8e3b-44f1-b5b7-22c7eb8bfcef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:30.261934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:30.263059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.263340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.263614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.264450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:30.266298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.286633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1086/2174] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:43:30.291031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:30.308131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd004d4-577a-4d27-85b6-96af3cf453c8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:30.320942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa266752-7ed7-4e3e-b8fc-fd7c12eb3a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:30.322099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa266752-7ed7-4e3e-b8fc-fd7c12eb3a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.322729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa266752-7ed7-4e3e-b8fc-fd7c12eb3a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.323253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa266752-7ed7-4e3e-b8fc-fd7c12eb3a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.324628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa266752-7ed7-4e3e-b8fc-fd7c12eb3a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:30.327202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa266752-7ed7-4e3e-b8fc-fd7c12eb3a86 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.347773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1089/2175] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:43:30.349802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:30.382099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc03d23-f05c-4626-9d62-a70b8c3651d5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:30.389585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:30.390692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.391240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.391780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] PATH_INFO: `/v3/projects/b377808342d440df9c2052da2283e16b/users/4de9d7bfe28d4b2c831873adef9345c3/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.425089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b377808342d440df9c2052da2283e16b, user_id=4de9d7bfe28d4b2c831873adef9345c3, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:30.434671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.464740 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1087/2176] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:43:30 2022] PUT /identity/v3/projects/b377808342d440df9c2052da2283e16b/users/4de9d7bfe28d4b2c831873adef9345c3/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:30.485660 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff1328-f4dd-4eb7-8395-617483ded2e6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.486129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff1328-f4dd-4eb7-8395-617483ded2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.486448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff1328-f4dd-4eb7-8395-617483ded2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.542361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2ff1328-f4dd-4eb7-8395-617483ded2e6 None None] MFA Rules not processed for user `4de9d7bfe28d4b2c831873adef9345c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:30.544156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2ff1328-f4dd-4eb7-8395-617483ded2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:30.657911 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1090/2177] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:30.676131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:30.679672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:30.833182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545ccd5-9694-4673-b902-68be16435fea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:30.840373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:30.841088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:30.841370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:30.841644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:30.854135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:30.855914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:30.875428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1088/2178] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:43:40.648969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:40.863030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:42.929502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367a03-1020-4f75-9802-3b7803cd2371 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:42.930128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367a03-1020-4f75-9802-3b7803cd2371 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:42.930439 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367a03-1020-4f75-9802-3b7803cd2371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:42.951298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5b367a03-1020-4f75-9802-3b7803cd2371 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:42.952545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5b367a03-1020-4f75-9802-3b7803cd2371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:43.028707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1091/2179] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:43.044194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.046060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:43.130088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d24c18-41c3-402a-97b2-713920d67ca0 service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.135263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.135945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.136357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.136716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.148449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:43.152228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:43.161491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1089/2180] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:43:43.793154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33f27-c0f8-43ae-a5d6-0779f28271cd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.793569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33f27-c0f8-43ae-a5d6-0779f28271cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.793944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d33f27-c0f8-43ae-a5d6-0779f28271cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.821316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b3d33f27-c0f8-43ae-a5d6-0779f28271cd None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:43.823074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b3d33f27-c0f8-43ae-a5d6-0779f28271cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:43.831340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1092/2181] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 25 14:43:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:43.846064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:43.857778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158f3c2-97d4-413e-b8bf-7bcb1870c3b7 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:43.863635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:43.864342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:43.864621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:43.864895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:43.882883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:43.885849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:43.904462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1090/2182] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 14:43:44.147994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d86d0-d4db-4676-9cf5-90940de705fb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:44.148650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d86d0-d4db-4676-9cf5-90940de705fb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:44.148650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d86d0-d4db-4676-9cf5-90940de705fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:44.176999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e08d86d0-d4db-4676-9cf5-90940de705fb None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:44.179010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e08d86d0-d4db-4676-9cf5-90940de705fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:44.188174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1093/2183] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 25 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:44.201648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:44.216226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375d4d3-414f-4453-9351-d99a5e67906e service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:44.223308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:44.224395 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:44.224878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:44.225349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:44.237386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:44.239186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:44.249138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1091/2184] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:43:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:43:44.718910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05666dc2-a45d-4e72-a2c5-ef96027f0114 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:44.719074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05666dc2-a45d-4e72-a2c5-ef96027f0114 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:44.719270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05666dc2-a45d-4e72-a2c5-ef96027f0114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:44.742949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-05666dc2-a45d-4e72-a2c5-ef96027f0114 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:44.745487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-05666dc2-a45d-4e72-a2c5-ef96027f0114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:44.761654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1094/2185] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:44.782393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:44.796009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad70c3c-4ead-4256-a586-99bc13fc8923 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:44.804234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:44.804954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:44.805256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:44.805529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:44.817863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:44.820009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:44.831142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1092/2186] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:43:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 14:43:45.260794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134221fd-c12c-47e0-b56b-a839c8f808c8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:45.261554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134221fd-c12c-47e0-b56b-a839c8f808c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:45.261554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134221fd-c12c-47e0-b56b-a839c8f808c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:45.291078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-134221fd-c12c-47e0-b56b-a839c8f808c8 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:45.292313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-134221fd-c12c-47e0-b56b-a839c8f808c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:45.299965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1095/2187] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 25 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:45.312973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:45.325359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec48cb-253f-4c9e-9c04-ff11d4f9906a service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:45.334730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852f098-72bb-4259-9d5b-96a6d0dd4eaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:45.335841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852f098-72bb-4259-9d5b-96a6d0dd4eaf service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:45.336319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852f098-72bb-4259-9d5b-96a6d0dd4eaf service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:45.336902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852f098-72bb-4259-9d5b-96a6d0dd4eaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:45.351585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852f098-72bb-4259-9d5b-96a6d0dd4eaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:45.353402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852f098-72bb-4259-9d5b-96a6d0dd4eaf service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:45.363505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1093/2188] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:43:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:43:46.526174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd45ea-a749-48a4-98de-d8368825404a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:46.527893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd45ea-a749-48a4-98de-d8368825404a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:46.528499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd45ea-a749-48a4-98de-d8368825404a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:46.531347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1096/2189] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:43:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:43:46.540489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:46.541609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:46.542240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:46.630338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd45ea-a749-48a4-98de-d8368825404a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:46.632569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-36fd45ea-a749-48a4-98de-d8368825404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:46.651495 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] MFA Rules not processed for user `7c1699703bdb4d2e8dac40a28792cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:46.654228 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:46.725655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd45ea-a749-48a4-98de-d8368825404a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:46.731466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:46.732532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:46.732808 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:46.733092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] PATH_INFO: `/v3/users/870b2f691e0145a6a28e541ad506af02` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:46.735153 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=870b2f691e0145a6a28e541ad506af02)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:46.737242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:46.760162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1094/2190] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:46.773295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:46.775927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:46.791102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad092-f5c2-47b6-83e2-84eaf9f9e7e5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:46.799673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:46.800118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:46.800408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:46.800687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:46.811143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:46.813002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:46.829504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1095/2191] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:43:46.829601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870b2f691e0145a6a28e541ad506af02'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:46.829920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870b2f691e0145a6a28e541ad506af02'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:46.830865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870b2f691e0145a6a28e541ad506af02'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:46.850645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870b2f691e0145a6a28e541ad506af02'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:46.859082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-598193be-c318-43d6-b569-33db4ca0b8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870b2f691e0145a6a28e541ad506af02'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:46.874252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:46.885214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1097/2192] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:43:46 2022] DELETE /identity/v3/users/870b2f691e0145a6a28e541ad506af02 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:47.133984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.148906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2f046-2186-4175-b8dd-f8c723a1a963 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.153220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:47.244948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.244948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.244948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.244948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] PATH_INFO: `/v3/projects/4b05021ee75a47f29be880e447488dcb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.246610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b05021ee75a47f29be880e447488dcb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.249662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.315697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b05021ee75a47f29be880e447488dcb', 'request_id': 'req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:47.325143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b05021ee75a47f29be880e447488dcb', 'request_id': 'req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:47.334160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b05021ee75a47f29be880e447488dcb', 'request_id': 'req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:47.352819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] The token cache is being invalidate because project 4b05021ee75a47f29be880e447488dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:43:47.353283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:47.354127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c7d820fd-4520-45fa-bcfa-dd8aac1270fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:47.357600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1096/2193] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:43:47 2022] DELETE /identity/v3/projects/4b05021ee75a47f29be880e447488dcb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:47.364003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.364358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.364635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.386855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:47.388197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:47.468672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1098/2194] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:47.476876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.477222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.477541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.497787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:47.499042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:47.508246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1097/2195] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:43:47.512917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.524033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7dee92-6b6e-4c7e-be6e-c1183793851a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.532031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.532900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.533038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.533269 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.534059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:43:47.534360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.536050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.548718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1099/2196] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:43:47.553415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.563818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70441a-4368-4754-b513-445dc84bc090 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.568829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.569432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.569714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.569988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.570709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.572771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.604697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1098/2197] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:43:47 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:43:47.611183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.622699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cf935-0e6e-4c84-8812-dc49c0f03833 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.687108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.687793 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.688079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.688355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.689045 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.691030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.718486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1100/2198] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:43:47 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:43:47.723500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.735144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a7b0f-76f2-4b4e-bb3e-ce98b0e63f49 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.741339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.742252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.742635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.742987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.743898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.745648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.758086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1099/2199] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:43:47.763975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.775594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a270aa-9657-481a-accc-135ab3398156 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.780979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.781664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.781952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.782381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.783350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.785298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.800854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1101/2200] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:43:47.805947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.822409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a329d3-5545-4b50-962f-95f5cb2caf50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.828703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b9433-aa53-4aa8-92d1-44097a845ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.829662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b9433-aa53-4aa8-92d1-44097a845ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.832474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b9433-aa53-4aa8-92d1-44097a845ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.832474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b9433-aa53-4aa8-92d1-44097a845ba9 admin admin] PATH_INFO: `/v3/projects/e385e42e642f445b814a2aa72fad1421/users/9e5050c0dc3f441f94d0c5e084088429/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.855068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b9433-aa53-4aa8-92d1-44097a845ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e385e42e642f445b814a2aa72fad1421, user_id=9e5050c0dc3f441f94d0c5e084088429, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.859893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b9433-aa53-4aa8-92d1-44097a845ba9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:47.873101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:47.875953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:47.879404 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1100/2201] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:43:47 2022] PUT /identity/v3/projects/e385e42e642f445b814a2aa72fad1421/users/9e5050c0dc3f441f94d0c5e084088429/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:47.894661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c462ae1-1950-4021-8210-d12a0db06410 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:47.895291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.895761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.896164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.930545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] MFA Rules not processed for user `9e5050c0dc3f441f94d0c5e084088429`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:47.932356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:47.956084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:47.987185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:47.988455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:47.988831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:47.989373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] PATH_INFO: `/v3/users/c34a0e6c8c424f7b97fef8059c5f3c96` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:47.991401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34a0e6c8c424f7b97fef8059c5f3c96)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:47.994649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:48.002713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:48.029372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1101/2202] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:48.041711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:48.043322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:48.074679 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34a0e6c8c424f7b97fef8059c5f3c96'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:48.075316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34a0e6c8c424f7b97fef8059c5f3c96'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:48.075943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34a0e6c8c424f7b97fef8059c5f3c96'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:48.091622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34a0e6c8c424f7b97fef8059c5f3c96'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:48.101901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eee4f646-e360-4044-81cd-065b56c647b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34a0e6c8c424f7b97fef8059c5f3c96'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:48.121610 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1102/2203] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:43:47 2022] DELETE /identity/v3/users/c34a0e6c8c424f7b97fef8059c5f3c96 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:43:48.128784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:48.128784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:48.128784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:48.131485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1cece-d0e3-4c9d-83ec-f35ad868ab80 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:48.154925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:43:48.156873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:48.206278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:48.208015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:48.208309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:48.208698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:48.213078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:48.250816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1103/2204] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:43:48.265702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:48.267367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:48.318360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:48.319886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:48.330955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1102/2205] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:43:48.356377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cc7fb-3f0b-400f-8a1c-fbb9395e65be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:48.360365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1420e-deb2-417d-9b17-2b507cebe2b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:48.361113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1420e-deb2-417d-9b17-2b507cebe2b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:48.361298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1420e-deb2-417d-9b17-2b507cebe2b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:48.361576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1420e-deb2-417d-9b17-2b507cebe2b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:48.372427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1420e-deb2-417d-9b17-2b507cebe2b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:48.373905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1420e-deb2-417d-9b17-2b507cebe2b7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:48.387724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1104/2206] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:43:48.938242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:43:48.940207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:43:48.963201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4e37-d0b5-4b02-8748-7d038363c202 service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:43:48.976804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:43:48.978802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:43:48.979153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:43:48.979434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] PATH_INFO: `/v3/projects/34e1c75d3f204befb221cbafb8fdae7c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:43:48.981859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e1c75d3f204befb221cbafb8fdae7c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:43:48.984164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:43:49.055204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:49.093860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e1c75d3f204befb221cbafb8fdae7c', 'request_id': 'req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:49.109515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e1c75d3f204befb221cbafb8fdae7c', 'request_id': 'req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:49.116667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e1c75d3f204befb221cbafb8fdae7c', 'request_id': 'req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:49.136492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] The token cache is being invalidate because project 34e1c75d3f204befb221cbafb8fdae7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:43:49.136797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:49.137554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:43:49.139230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1103/2207] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:43:48 2022] DELETE /identity/v3/projects/34e1c75d3f204befb221cbafb8fdae7c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:43:58.379792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=77,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:43:59.134604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:47.952066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a5db0-41b3-4c4c-9265-6487c7e6aa72 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:47.952702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a5db0-41b3-4c4c-9265-6487c7e6aa72 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:47.952702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a5db0-41b3-4c4c-9265-6487c7e6aa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:47.990080 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f17a5db0-41b3-4c4c-9265-6487c7e6aa72 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:47.991470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f17a5db0-41b3-4c4c-9265-6487c7e6aa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:48.089607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1105/2208] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:48.179300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:48.185223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:48.309332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea66d-0ce4-4432-b7ac-d4b1211ae0f6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:48.317428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:48.318701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:48.319167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:48.319568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:48.334316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:48.336988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:48.362503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1104/2209] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:44:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:44:48.424027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac793-abe4-4b1a-8373-12394abdb98f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:48.424293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac793-abe4-4b1a-8373-12394abdb98f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:48.424431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eac793-abe4-4b1a-8373-12394abdb98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:48.454682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-04eac793-abe4-4b1a-8373-12394abdb98f None None] MFA Rules not processed for user `2401371a775743c29c16406cb9df5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:48.456701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04eac793-abe4-4b1a-8373-12394abdb98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:48.570176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1106/2210] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:50.294587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:50.297222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:50.492654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4faec5-5171-4709-bcb1-80956eb36869 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:50.500276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:50.502027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:50.502614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:50.503256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] PATH_INFO: `/v3/users/9e5050c0dc3f441f94d0c5e084088429` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:50.506253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5050c0dc3f441f94d0c5e084088429)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:50.510118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:50.642305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5050c0dc3f441f94d0c5e084088429'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:50.643812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5050c0dc3f441f94d0c5e084088429'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:50.665919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5050c0dc3f441f94d0c5e084088429'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:50.667586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5050c0dc3f441f94d0c5e084088429'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:50.686741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5050c0dc3f441f94d0c5e084088429'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:50.721877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1105/2211] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:44:50 2022] DELETE /identity/v3/users/9e5050c0dc3f441f94d0c5e084088429 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:44:50.728256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:50.729144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:50.729144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:50.787156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:50.787156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:50.901724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1107/2212] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:50.918536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:50.921158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:50.964613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:51.072294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57764701-4ae8-482d-88b5-a26dda3f03c9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:51.076723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794eeca-754f-4df3-b216-df0529182a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:51.077596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794eeca-754f-4df3-b216-df0529182a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:51.078181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794eeca-754f-4df3-b216-df0529182a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:51.078800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794eeca-754f-4df3-b216-df0529182a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:51.105093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794eeca-754f-4df3-b216-df0529182a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:51.107670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794eeca-754f-4df3-b216-df0529182a78 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:51.140407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1106/2213] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:44:51.674825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:51.676401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:51.716771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d68ef-51ab-4be7-84d3-a7eb4e8399ce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:51.746807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:51.754946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:51.755414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:51.756495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] PATH_INFO: `/v3/projects/e385e42e642f445b814a2aa72fad1421` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:51.761161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e385e42e642f445b814a2aa72fad1421)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:51.768108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:51.943258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:51.954193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e385e42e642f445b814a2aa72fad1421', 'request_id': 'req-03f05d53-ca7b-4427-8365-3959fdda3522'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:51.970931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e385e42e642f445b814a2aa72fad1421', 'request_id': 'req-03f05d53-ca7b-4427-8365-3959fdda3522'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:51.990152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e385e42e642f445b814a2aa72fad1421', 'request_id': 'req-03f05d53-ca7b-4427-8365-3959fdda3522'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:52.033193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] The token cache is being invalidate because project e385e42e642f445b814a2aa72fad1421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:44:52.033732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:52.034916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-03f05d53-ca7b-4427-8365-3959fdda3522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:52.043209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1108/2214] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:44:51 2022] DELETE /identity/v3/projects/e385e42e642f445b814a2aa72fad1421 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:44:52.052221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.053940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.054745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.099182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:52.100548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:52.280942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1107/2215] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:52.294963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.295463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.295961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.338378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:52.348202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:52.374789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1109/2216] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:52.379828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.404287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d53cba-6d01-46e9-95eb-21366aaaefe4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.424679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.425722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.426097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.426617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.428418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:52.428724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.430978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.462161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1108/2217] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:52.471991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.485652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead5de2-3280-48ca-b8b1-50321488378c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.494066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.495458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.495678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.496307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.498596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.505005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.574315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1110/2218] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:44:52 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:44:52.584670 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.605259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e30e9e-3317-415e-93d0-6bcad79d5ff0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.649923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:52.653536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:52.900036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6143d0-5fbd-467e-92b2-61eaa0a2be63 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:52.914566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.915856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.916407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.917051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.918337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.923151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:52.925021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.925729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:52.926281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:52.927207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] PATH_INFO: `/v3/users/4de9d7bfe28d4b2c831873adef9345c3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:52.935763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de9d7bfe28d4b2c831873adef9345c3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:52.935763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:52.998323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1109/2219] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:44:52 2022] POST /identity/v3/users => generated 291 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:44:53.002617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:53.022609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45014d61-a3bd-4a0f-bb54-6501044119a7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:53.029961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:53.030871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.031287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.031640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.032641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:53.035102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:53.057325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1110/2220] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:44:53.058515 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:53.069872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de9d7bfe28d4b2c831873adef9345c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:53.070935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de9d7bfe28d4b2c831873adef9345c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:53.071488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c7bae-6db4-492b-88d2-b1dfe0e4bb24 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:53.076884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de9d7bfe28d4b2c831873adef9345c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:53.078262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:53.079658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.080603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.081054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.082110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:53.084761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:53.107509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de9d7bfe28d4b2c831873adef9345c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:53.108350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1111/2221] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:44:53.116456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:53.119939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:53.121792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a70a0cef-237a-4b7a-b3ba-9875cddbccb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de9d7bfe28d4b2c831873adef9345c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:53.148175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29854b0-5f52-4bbb-aaf3-fe156d2403fa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:53.159296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d5689-4372-45f3-96e0-5d4d7e10129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:53.161329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5689-4372-45f3-96e0-5d4d7e10129f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.162031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5689-4372-45f3-96e0-5d4d7e10129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.162321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1111/2222] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:44:52 2022] DELETE /identity/v3/users/4de9d7bfe28d4b2c831873adef9345c3 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:44:53.162650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5689-4372-45f3-96e0-5d4d7e10129f admin admin] PATH_INFO: `/v3/projects/f0ac9e06d91046f7b46405794ea1203f/users/d43b4b2c5433485bbf8457eeea4e97b3/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.168357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:53.170340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.170863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.173132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.209492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d5689-4372-45f3-96e0-5d4d7e10129f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ac9e06d91046f7b46405794ea1203f, user_id=d43b4b2c5433485bbf8457eeea4e97b3, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:53.212321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d5689-4372-45f3-96e0-5d4d7e10129f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:53.230715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:53.230715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:53.245168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1112/2223] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:44:53 2022] PUT /identity/v3/projects/f0ac9e06d91046f7b46405794ea1203f/users/d43b4b2c5433485bbf8457eeea4e97b3/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:44:53.260157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.260623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.261329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.282792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:53.299314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:53.303206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:53.380977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1112/2224] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:53.396054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:53.398284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:53.553940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1113/2225] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:53.580011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:53.584671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:53.589063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0e803-c60c-44b5-b549-fbbcc275e8e0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:53.594244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:53.595035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.595316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.595624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.605353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5b77b-2c9f-4981-aa86-04251a86ca6e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:53.615871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:53.619069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:53.628737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81744976-8bd8-45b7-ba4f-9694b6ed5376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:53.629598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81744976-8bd8-45b7-ba4f-9694b6ed5376 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:53.629977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81744976-8bd8-45b7-ba4f-9694b6ed5376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:53.630344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81744976-8bd8-45b7-ba4f-9694b6ed5376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:53.645012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1113/2226] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:44:53.653367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81744976-8bd8-45b7-ba4f-9694b6ed5376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:53.655540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81744976-8bd8-45b7-ba4f-9694b6ed5376 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:53.667976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1114/2227] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:44:54.730566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:54.732549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:54.768691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab004b-be1f-498b-83d9-04a6fe7344eb service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:54.792999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:54.794448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:54.797154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:54.799068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] PATH_INFO: `/v3/projects/b377808342d440df9c2052da2283e16b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:54.805968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b377808342d440df9c2052da2283e16b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:54.813192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:54.986743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b377808342d440df9c2052da2283e16b', 'request_id': 'req-0dcedbec-8156-41d0-b82f-518ef69614ed'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:55.020674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b377808342d440df9c2052da2283e16b', 'request_id': 'req-0dcedbec-8156-41d0-b82f-518ef69614ed'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:55.041588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b377808342d440df9c2052da2283e16b', 'request_id': 'req-0dcedbec-8156-41d0-b82f-518ef69614ed'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:55.090189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] The token cache is being invalidate because project b377808342d440df9c2052da2283e16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:44:55.090189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:55.091879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.092338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.092731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.097431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0dcedbec-8156-41d0-b82f-518ef69614ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:44:55.107779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1114/2228] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:44:54 2022] DELETE /identity/v3/projects/b377808342d440df9c2052da2283e16b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:44:55.110120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153387e7-56a9-41c3-9b79-5f5fa25a5f55 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.110391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153387e7-56a9-41c3-9b79-5f5fa25a5f55 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.110825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153387e7-56a9-41c3-9b79-5f5fa25a5f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.170150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] MFA Rules not processed for user `d43b4b2c5433485bbf8457eeea4e97b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:55.170923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-153387e7-56a9-41c3-9b79-5f5fa25a5f55 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:55.171910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:55.172245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-153387e7-56a9-41c3-9b79-5f5fa25a5f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:55.339753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:55.368757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1115/2229] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:55.378161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.378436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1115/2230] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:44:55.378609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.379055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.402288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.405369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:55.414223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:55.415872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:55.432647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1116/2231] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:44:55.434112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.449782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b3c9f-aca6-492f-a1a7-85260e45afbc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.464349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.465088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.465372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.465641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.466621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:44:55.467016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.468848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.496408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1117/2232] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 14:44:55.500705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.519606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2337f3-4802-424c-9f10-75cbbb9268af admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.528657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.530338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.530681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.531063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.531876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.535186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.568765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bc119-f28b-4e7e-ab01-759dd0cb8f02 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.574075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.574711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.575221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.575384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.586343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.595055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1118/2233] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:44:55 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 14:44:55.599263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.600429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.625833 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1116/2234] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:44:55.725454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a137da-2dc9-4a5d-a631-c6ffcab1db2b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.734442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.735803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.736298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.737150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.738438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.742143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.778568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1119/2235] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:44:55 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:44:55.783182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.790270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:55.801687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25fe2c-28c7-4714-87a2-f050ba04c2e5 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.813069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.815272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.815686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.815974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.817139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.818481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.846693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1117/2236] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:44:55.849395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.870799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71e37b-01dd-4ed4-91b2-9854bedd8685 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.878995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7915364-a690-4850-83af-2e39c14e397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.880976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7915364-a690-4850-83af-2e39c14e397c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.881407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7915364-a690-4850-83af-2e39c14e397c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.881751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7915364-a690-4850-83af-2e39c14e397c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.882865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7915364-a690-4850-83af-2e39c14e397c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.885277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7915364-a690-4850-83af-2e39c14e397c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:55.910800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1120/2237] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:44:55.916855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:55.935555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74051d-6faa-4bff-9d00-6824f348279b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:55.943895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:55.945839 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:55.946673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:55.947438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] PATH_INFO: `/v3/projects/8fef605d3e684049920ea9271ad6d352/users/5b382154ad2845fc8b07bcd8dc2a8b78/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:55.977480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fef605d3e684049920ea9271ad6d352, user_id=5b382154ad2845fc8b07bcd8dc2a8b78, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:55.981366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.023298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1118/2238] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:44:55 2022] PUT /identity/v3/projects/8fef605d3e684049920ea9271ad6d352/users/5b382154ad2845fc8b07bcd8dc2a8b78/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:44:56.026952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8a93c-e25d-4391-8591-10b373f14513 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.027309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8a93c-e25d-4391-8591-10b373f14513 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.027589 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8a93c-e25d-4391-8591-10b373f14513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.072690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c9d8a93c-e25d-4391-8591-10b373f14513 None None] MFA Rules not processed for user `5b382154ad2845fc8b07bcd8dc2a8b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:56.074855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c9d8a93c-e25d-4391-8591-10b373f14513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:56.360210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1121/2239] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:44:56.369883 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:56.372306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:56.484993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e263c-a2cf-4ee7-9b29-cb7b06d31aaa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:56.489185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:56.489871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:56.490171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:56.490442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:56.504484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:56.507874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:44:56.528731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1119/2240] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:44:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:44:57.378892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7a85-eb85-4fc8-82fd-255bd806f582 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:57.378892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7a85-eb85-4fc8-82fd-255bd806f582 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:57.378892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7a85-eb85-4fc8-82fd-255bd806f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:57.412651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6b7c7a85-eb85-4fc8-82fd-255bd806f582 None None] MFA Rules not processed for user `d43b4b2c5433485bbf8457eeea4e97b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:44:57.414602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6b7c7a85-eb85-4fc8-82fd-255bd806f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:44:57.434190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:44:57.566810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1122/2241] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:44:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:44:59.677163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:44:59.898572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b63514-6c70-4f1b-9d07-ec334d9bebbf service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:44:59.908823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:44:59.910219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:44:59.911098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:44:59.911530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] PATH_INFO: `/v3/users/5b382154ad2845fc8b07bcd8dc2a8b78` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:44:59.914093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b382154ad2845fc8b07bcd8dc2a8b78)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:44:59.917755 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:00.090040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b382154ad2845fc8b07bcd8dc2a8b78'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:00.096180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b382154ad2845fc8b07bcd8dc2a8b78'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:00.113742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b382154ad2845fc8b07bcd8dc2a8b78'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:00.115421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b382154ad2845fc8b07bcd8dc2a8b78'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:00.159226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:00.183142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b382154ad2845fc8b07bcd8dc2a8b78'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:00.231101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1120/2242] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:44:59 2022] DELETE /identity/v3/users/5b382154ad2845fc8b07bcd8dc2a8b78 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:45:00.237653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.238377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.238377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.278519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:00.282809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:00.504530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1123/2243] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:00.520728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:00.523935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:00.755167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bf598-02fe-4cae-8143-d7f75ffd904a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:00.760931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b283-bcb3-4646-92a0-1bc8dba6f22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:00.762132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b283-bcb3-4646-92a0-1bc8dba6f22b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:00.762651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b283-bcb3-4646-92a0-1bc8dba6f22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:00.763000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b283-bcb3-4646-92a0-1bc8dba6f22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:00.791724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019b283-bcb3-4646-92a0-1bc8dba6f22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:00.797069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019b283-bcb3-4646-92a0-1bc8dba6f22b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:00.826240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1121/2244] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:45:01.849658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:01.852102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:01.867803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428499-86fc-4899-832d-1ef7e0136b6f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:01.875682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:01.877133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:01.877663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:01.878204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] PATH_INFO: `/v3/projects/8fef605d3e684049920ea9271ad6d352` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:01.879953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fef605d3e684049920ea9271ad6d352)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:01.882500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.089396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fef605d3e684049920ea9271ad6d352', 'request_id': 'req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:02.107630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fef605d3e684049920ea9271ad6d352', 'request_id': 'req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:02.136056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fef605d3e684049920ea9271ad6d352', 'request_id': 'req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:02.172118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] The token cache is being invalidate because project 8fef605d3e684049920ea9271ad6d352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:45:02.172118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:02.173186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19f0a524-6fa9-4286-9dc4-2c81f5dfc0e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:45:02.184444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.185201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.185201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.195379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1124/2245] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:45:01 2022] DELETE /identity/v3/projects/8fef605d3e684049920ea9271ad6d352 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 14:45:02.221301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:02.228943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:02.409054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1122/2246] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:02.412274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.413022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.413750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.470257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:02.472798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:02.495590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1125/2247] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:02.495915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.520444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e743e5-bb78-4770-8b4b-b94e4fa9a84c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.530949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.532263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.533724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.534385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.536501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 14:45:02.537811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.541693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.574801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1123/2248] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 14:45:02.581547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.598481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1ea11-d75a-4598-8039-e59890e6559c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.612188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.613113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.613466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.613798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.615378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.617907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.687421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1126/2249] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 14:45:02 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:45:02.688963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.705300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33eed6-26dc-42d5-9bef-215f2d46f375 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.850709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.855845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.856540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.856984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.860035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.863617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.912131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1124/2250] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 14:45:02 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 14:45:02.913523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:02.942114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81874c-2ae6-4650-b05e-b17898e9b2e7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:02.951378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:02.952343 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:02.952719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:02.953168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:02.954630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:02.956977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:02.980776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1127/2251] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:45:02.991427 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:03.016467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba924cc3-5940-49bd-a264-89c4cb7d33f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:03.032366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5048df-1196-4bc1-a570-6465144ed000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:03.035508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5048df-1196-4bc1-a570-6465144ed000 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:03.036007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5048df-1196-4bc1-a570-6465144ed000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:03.036497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5048df-1196-4bc1-a570-6465144ed000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:03.038479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5048df-1196-4bc1-a570-6465144ed000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:03.050679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5048df-1196-4bc1-a570-6465144ed000 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:03.060471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:03.089560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1125/2252] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 14:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 14:45:03.091824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:03.112331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b7c3-748a-427c-8912-5b89ca4e4850 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:03.121946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:03.125576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:03.126135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:03.126787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] PATH_INFO: `/v3/projects/2394f31a9792405db254d9039a41cae2/users/a08ca3198f59463fa32872d40bb5d0d3/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:03.132216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:03.181727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2394f31a9792405db254d9039a41cae2, user_id=a08ca3198f59463fa32872d40bb5d0d3, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:03.196792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:03.248254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1128/2253] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 14:45:03 2022] PUT /identity/v3/projects/2394f31a9792405db254d9039a41cae2/users/a08ca3198f59463fa32872d40bb5d0d3/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:45:03.263730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:03.264572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:03.264572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:03.350444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] MFA Rules not processed for user `a08ca3198f59463fa32872d40bb5d0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:03.352215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:03.493036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1126/2254] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:45:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 14:45:03.522709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:03.525902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:03.618026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe947-09ac-4f3a-bc02-8117e8ce9500 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:03.626550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:03.628085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:03.628577 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:03.629015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:03.650305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:03.652419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:03.673656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1129/2255] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:04.320738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:04.324033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:04.447552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961765b2-bf88-4745-8cb3-14789f332343 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:04.456912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:04.458536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:04.459352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:04.460369 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:04.464169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:04.659360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:04.661079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52889156-ee23-4b23-b0f1-e773a1b42fab service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:04.700067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1127/2256] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:05.061308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:05.063208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:05.093477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc956e3f-8c99-4e33-a601-eb3fced57bc8 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:05.097613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:05.098638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:05.098949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:05.099223 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:05.101130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:05.117974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:05.121030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0784-269a-40a5-87b4-4ae73340e733 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:05.142084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1130/2257] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:14.187494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14890ac4-be6d-423f-b6f4-5847ccaa1569 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.188137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14890ac4-be6d-423f-b6f4-5847ccaa1569 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.188238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14890ac4-be6d-423f-b6f4-5847ccaa1569 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.189674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1128/2258] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 14:45:14.198165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11368c-a743-482a-8ee1-29d3d02f3f61 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.198981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11368c-a743-482a-8ee1-29d3d02f3f61 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.198981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11368c-a743-482a-8ee1-29d3d02f3f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.261135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8f11368c-a743-482a-8ee1-29d3d02f3f61 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:14.264226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8f11368c-a743-482a-8ee1-29d3d02f3f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:14.283801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1131/2259] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:14.300738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14890ac4-be6d-423f-b6f4-5847ccaa1569 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.302629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-14890ac4-be6d-423f-b6f4-5847ccaa1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:14.316256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:14.382336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14890ac4-be6d-423f-b6f4-5847ccaa1569 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.386498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3e5b1-38b7-4b55-90e8-bfc453556382 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.387316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3e5b1-38b7-4b55-90e8-bfc453556382 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.387687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3e5b1-38b7-4b55-90e8-bfc453556382 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.388076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3e5b1-38b7-4b55-90e8-bfc453556382 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.402488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3e5b1-38b7-4b55-90e8-bfc453556382 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:14.404257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3e5b1-38b7-4b55-90e8-bfc453556382 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:14.418310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1129/2260] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:14.506595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79723399-8798-46aa-b2fb-805f67e39b59 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.507114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79723399-8798-46aa-b2fb-805f67e39b59 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.507586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79723399-8798-46aa-b2fb-805f67e39b59 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.510154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1132/2261] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:45:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:14.518712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b36897-241d-4a5d-81ae-1095187c8282 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.519072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b36897-241d-4a5d-81ae-1095187c8282 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.519351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b36897-241d-4a5d-81ae-1095187c8282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.547541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e7b36897-241d-4a5d-81ae-1095187c8282 None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:14.549679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7b36897-241d-4a5d-81ae-1095187c8282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:14.562548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1130/2262] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:14.581234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79723399-8798-46aa-b2fb-805f67e39b59 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.595945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79723399-8798-46aa-b2fb-805f67e39b59 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.599698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4383b-49f5-4617-88c4-2c43f3224ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.600367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4383b-49f5-4617-88c4-2c43f3224ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.600648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4383b-49f5-4617-88c4-2c43f3224ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.600922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4383b-49f5-4617-88c4-2c43f3224ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.610403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4383b-49f5-4617-88c4-2c43f3224ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:14.612079 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4383b-49f5-4617-88c4-2c43f3224ba5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:14.621503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1133/2263] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:45:14.733287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f3195-9210-4699-84a9-677c938fa7f3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.733774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f3195-9210-4699-84a9-677c938fa7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.734245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f3195-9210-4699-84a9-677c938fa7f3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.736843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1131/2264] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 14:45:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 14:45:14.745917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.746476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.747039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.776064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:14.777652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:14.788984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1134/2265] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:14.805424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f3195-9210-4699-84a9-677c938fa7f3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.833047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f3195-9210-4699-84a9-677c938fa7f3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:14.839211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96085db8-7781-4029-8281-965112566d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:14.840304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96085db8-7781-4029-8281-965112566d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:14.840779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96085db8-7781-4029-8281-965112566d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:14.841248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96085db8-7781-4029-8281-965112566d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:14.858941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96085db8-7781-4029-8281-965112566d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:14.861408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96085db8-7781-4029-8281-965112566d9c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:14.881664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1132/2266] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:14.977813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:14.980066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:15.076874 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f872a-f2bb-4246-b535-96a4e34de566 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.085303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.086333 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.086834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.087302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.090152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:15.107373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.109097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.124746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1135/2267] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:15.457566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96085db8-7781-4029-8281-965112566d9c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:15.459919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-96085db8-7781-4029-8281-965112566d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:15.488122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96085db8-7781-4029-8281-965112566d9c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:15.497380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:15.498081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:15.498360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:15.498636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:15.500573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:15.600797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:15.603527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:15.621449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1133/2268] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:45:23.105230 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:23.123785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81e092-ccf2-4f2e-9763-eab77255e5f3 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:23.131422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:23.131826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:23.132583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:23.133282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:23.136796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:23.142436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:23.157210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf23d3-587b-4c2b-9c73-1c6e4562b249 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:23.162593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:23.166245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d22841-cfd9-470c-9c48-a3c8a4b7ade7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:23.168764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:23.169550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:23.169881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:23.170147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:23.184671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1136/2269] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:23.185481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:23.190978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:23.219699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1134/2270] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:45:33.175088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:33.205188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=65 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:37.581556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:37.582043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:37.582146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:37.600466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:45:37.601698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:37.662935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1137/2271] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:45:37.677875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:37.681119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:37.759045 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90ec64-c382-4107-be41-071c168634a5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:37.765423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:37.766009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:37.766292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:37.766593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:37.775808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:37.777455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:37.787288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1135/2272] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:45:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:45:47.655359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:47.783065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:45:52.947314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:45:52.961705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015f83d-7963-4ca3-a946-607d00e3acda None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:45:52.967830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:45:52.968494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:45:52.968769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:45:52.969055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:45:52.971114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:45:53.036723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:45:53.038340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e20b8-3228-486f-abe7-ee969daf04a5 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:45:53.047995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1138/2273] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:03.043752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:07.801379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:07.903918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5619d-f7a8-48b9-a09f-6032124b4a0e service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:07.910488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:07.911421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:07.911698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:07.911971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] PATH_INFO: `/v3/users/a08ca3198f59463fa32872d40bb5d0d3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:07.913233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08ca3198f59463fa32872d40bb5d0d3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:07.915199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:07.972342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08ca3198f59463fa32872d40bb5d0d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:07.973186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08ca3198f59463fa32872d40bb5d0d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:07.984599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08ca3198f59463fa32872d40bb5d0d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:07.985528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08ca3198f59463fa32872d40bb5d0d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.000224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08ca3198f59463fa32872d40bb5d0d3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.017182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1136/2274] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 14:46:07 2022] DELETE /identity/v3/users/a08ca3198f59463fa32872d40bb5d0d3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:46:08.022409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:08.023286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:08.023286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:08.043317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:08.044748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:08.115741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1139/2275] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:08.128456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:08.129959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:08.220712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d4fa4-39bc-40b7-a201-122814eebe47 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:08.227083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:08.228190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:08.228671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:08.229148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:08.245202 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:08.247978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:08.265746 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1137/2276] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 14:46:08.739274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:08.741735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:08.757339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177be63-d182-478f-8600-787fc7edb9b3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:08.762243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:08.763464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:08.763746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:08.764022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] PATH_INFO: `/v3/projects/2394f31a9792405db254d9039a41cae2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:08.765288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2394f31a9792405db254d9039a41cae2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:08.767731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:08.831785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2394f31a9792405db254d9039a41cae2', 'request_id': 'req-8c18e3e0-9397-44be-b4ea-49bc865f0856'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.843736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2394f31a9792405db254d9039a41cae2', 'request_id': 'req-8c18e3e0-9397-44be-b4ea-49bc865f0856'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.864594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2394f31a9792405db254d9039a41cae2', 'request_id': 'req-8c18e3e0-9397-44be-b4ea-49bc865f0856'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.893090 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] The token cache is being invalidate because project 2394f31a9792405db254d9039a41cae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 14:46:08.893639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.895204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8c18e3e0-9397-44be-b4ea-49bc865f0856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 14:46:08.898160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1140/2277] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 14:46:08 2022] DELETE /identity/v3/projects/2394f31a9792405db254d9039a41cae2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 14:46:14.506304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:14.507909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:14.552311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:14.636416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96910b-ff31-4c31-994e-f748c45732ad service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:14.719916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:14.720576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:14.720846 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:14.721109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:14.723016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:14.737278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:14.738910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:14.757880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1138/2278] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:17.452179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1cbc-3374-4ca3-835e-08fc14627929 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:17.453361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1cbc-3374-4ca3-835e-08fc14627929 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:17.459033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b1cbc-3374-4ca3-835e-08fc14627929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:17.483051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d71b1cbc-3374-4ca3-835e-08fc14627929 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:17.485809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d71b1cbc-3374-4ca3-835e-08fc14627929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:17.576923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1141/2279] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 25 14:46:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:17.607529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:17.609281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:17.693746 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315c6b-6181-487c-ae7e-6cbbea9fe5a8 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:17.698976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:17.699634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:17.699909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:17.700172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:17.710826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:17.713857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:17.723874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1139/2280] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 14:46:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 14:46:27.565811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:27.719876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:46:45.335488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2184-f370-4621-bf8b-30fc5d3f24c3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:45.336204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2184-f370-4621-bf8b-30fc5d3f24c3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:45.336204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2184-f370-4621-bf8b-30fc5d3f24c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:45.354675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-17fd2184-f370-4621-bf8b-30fc5d3f24c3 None None] MFA Rules not processed for user `73d50022c5f94444afc67ea810e4d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:46:45.355928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-17fd2184-f370-4621-bf8b-30fc5d3f24c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:45.433885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1142/2281] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:46:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:46:52.155053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:46:52.156777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:52.175929 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a484a-d519-46ef-9ddf-c3cde59ad963 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:46:52.184073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:46:52.184789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:46:52.185030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:46:52.185331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:46:52.187802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:46:52.210536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:46:52.212375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:46:52.226554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1140/2282] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:46:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:46:55.425066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:02.220141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:05.640518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:05.641078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:05.641078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:05.662156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] MFA Rules not processed for user `d43b4b2c5433485bbf8457eeea4e97b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 14:47:05.663448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:05.727312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1143/2283] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 14:47:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 14:47:05.983646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:05.985803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:06.046356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02048970-8ba7-418a-b932-4f56f34fe343 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:06.056467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:06.057159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:06.057438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:06.057711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:06.060795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:06.074538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:06.076089 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:06.084762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1141/2284] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:47:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:15.719145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:16.080959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:21.614279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:21.616142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:21.634214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9131-6c24-401d-9a70-6d0c68849ebd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:21.640921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:21.641592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:21.641875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:21.642147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:21.644047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:21.709349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:21.711037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:21.721491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1144/2285] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:47:31.716596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:47:50.246180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:47:50.261015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01aa1c-dd29-4c37-8721-3dcc0bfc323b service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:47:50.268251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:47:50.269274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:47:50.269736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:47:50.270208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:47:50.273321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:47:50.341651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:47:50.343484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:47:50.353981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1142/2286] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:47:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:00.348699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:48:19.230054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:48:19.231871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:19.321615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d3a68-68c2-4539-b7e1-4743b9381d95 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:48:19.331779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:48:19.332848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:48:19.333317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:48:19.333768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:48:19.337010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:48:19.365927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:48:19.369369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:48:19.387250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1145/2287] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:48:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:48:29.381590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=32 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:49:55.354618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:49:55.356217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:55.419751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a469a0-157f-4bfe-a079-1af6a096a45c service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:49:55.423806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:49:55.424436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:49:55.424711 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:49:55.424949 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:49:55.426932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:49:55.438577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:49:55.440031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:49:55.449254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1143/2288] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:49:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:50:05.445181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:15.216880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:15.231819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f1759-0e83-4443-8e86-94a4bc129f1a service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:15.238420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:15.239118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:15.239395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:15.239669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:15.241521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:15.329640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:15.331363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:15.351645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1146/2289] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:50:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:50:25.348633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=41 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:50:52.770954 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:50:52.784689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ccff3-35a6-4201-96a7-298071fbdfae service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:50:52.791659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:50:52.792424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:50:52.792744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:50:52.793039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:50:52.794973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:50:52.831308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:50:52.832944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:50:52.853175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1144/2290] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:50:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:51:02.850444 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=19 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:51:14.364632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:51:14.375500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf11e10-c598-4cc0-9793-d7300936dfee service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:51:14.379943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:51:14.380597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:51:14.380899 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:51:14.381179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:51:14.383600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:51:14.394863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:51:14.396423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:51:14.405169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1147/2291] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:51:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:51:24.401182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:52:21.664541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:21.687911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc49fc-f493-462f-82f8-0ea3fb89d3ce service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:21.692116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:21.692799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:21.693080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:21.693358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:21.717793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:21.719317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:21.749102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1145/2292] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:52:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:31.746277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=24 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:52:49.494040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:52:49.535630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc7f5c-6fb3-43d9-8890-abac235706bd service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:52:49.545536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:52:49.546494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:52:49.546817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:52:49.547234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:52:49.558405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:52:49.560035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:52:49.587572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1148/2293] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:52:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:52:59.585143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=21 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:53:27.631128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:53:27.661390 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7708a0-d8b1-494d-b50c-6a8c13433a7f service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:53:27.665868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:53:27.666492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:53:27.666773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:53:27.667074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:53:27.668978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:53:27.699336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:53:27.700760 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:53:27.711907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1146/2294] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:53:37.707449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=20 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:15.708040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:15.709979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:54:15.759983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6769a63-c995-4c33-87a1-5959467682ac service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:15.765804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:15.766456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:15.766733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:15.767033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:15.776501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:15.778446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:15.792705 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1149/2295] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:54:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:54:25.784386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=15 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:54:55.773432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:54:55.782927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f03cf-a12e-4188-b06d-f09f10d3b8c8 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:54:55.787535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:54:55.788208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:54:55.788494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:54:55.788762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:54:55.798869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:54:55.800354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:54:55.829618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1147/2296] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:54:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:05.826535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=13 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:15.975504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:15.985963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10618cd-eaf5-4262-a630-68567e1dd7c5 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:15.990625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:15.991301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:15.991583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:15.991853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:16.001672 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:16.003185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:16.013070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1150/2297] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:55:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:55:26.008514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:55:53.336665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:55:53.345721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698445ea-fe56-416e-aee2-889042272d73 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:55:53.349703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:55:53.350347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:55:53.350741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:55:53.350937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:55:53.360785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:55:53.362214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:55:53.371516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1148/2298] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:55:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:03.367139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:56:20.359907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:56:20.381115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3a7f8-3fc7-43f6-a23a-5255b29d9927 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:56:20.385677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:56:20.386330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:56:20.386588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:56:20.386902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:56:20.394964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:56:20.396403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:56:20.405326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1151/2299] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:56:30.400954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=10 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:21.832197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:21.834017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:21.941442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe4649-1d1e-4ac2-8d06-50c458593ef1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:21.947097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:21.947746 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:21.948021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:21.948290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:21.950268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:22.024748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:22.026370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:22.035550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1149/2300] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:57:32.031430 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=70 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:57:50.074019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:57:50.076662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:57:50.103602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd82e2b-bc0d-4efa-81ae-c5f5a5c782f9 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:57:50.107951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:57:50.108596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:57:50.108872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:57:50.109141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:57:50.118414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:57:50.119882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:57:50.129264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1152/2301] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:58:00.124836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=11 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:16.700009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:16.701825 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:16.786551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d14b9-b58c-4aed-b64b-1a35d8f22d23 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:16.792145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:16.792793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:16.793098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:16.793362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:16.795407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:16.808018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:16.809591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:16.822265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1150/2302] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:26.816184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:28.075113 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:28.076810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:28.151267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a992f7-ed75-40fc-8402-496417211655 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:28.155380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:28.156025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:28.156307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:28.156581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:28.167137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:28.168571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:28.179492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1153/2303] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:29.379841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:29.382205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:29.396960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f1a00-e00f-466e-975f-54b0b5684326 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:29.401119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:29.401768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:29.402032 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:29.402297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:29.404411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:29.419094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:29.420537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:29.432479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1151/2304] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:59:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 14:59:38.175152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:39.427347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 14:59:56.543972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 14:59:56.562120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854008-9566-43d8-9f44-d33b9d3dded1 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 14:59:56.572005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 14:59:56.572956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 14:59:56.573324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 14:59:56.573714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 14:59:56.576027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 14:59:56.648655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 14:59:56.650314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 14:59:56.659284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1154/2305] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 14:59:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:06.655399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:16.551972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:16.553691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:16.568326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7101e-da65-455e-86ef-ad992d653acf service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:16.575521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:16.576242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:16.576589 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:16.576859 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:16.578964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:00:16.664117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:16.665704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:16.684475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1152/2306] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:00:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:00:26.681580 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=41 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:00:52.654355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:00:52.673457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdb51c-28df-46ef-9361-b637546f5e7b service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:00:52.681752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:00:52.682404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:00:52.682680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:00:52.682983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:00:52.740821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:00:52.742425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:00:52.751832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1155/2307] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:00:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:01:02.747543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:21.981418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:22.039082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c631558-9f10-4d3e-a37f-37ab4607ef48 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:22.043464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:22.044114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:22.044391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:22.044662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:22.067479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:22.068912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:22.096567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1153/2308] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:29.371571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:29.388283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72bd8-4ef8-44ca-8609-87fd043f8f84 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:29.392525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:29.394094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:29.394094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:29.394094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:29.396058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:29.407206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:29.408655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:29.417533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1156/2309] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:32.093751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=39 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:39.413627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:02:49.389977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:02:49.403782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17974c-54da-4267-a584-3897c1d22fe0 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:02:49.410397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:02:49.410994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:02:49.411278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:02:49.411547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:02:49.413577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:02:49.485618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:02:49.487261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:02:49.496663 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1154/2310] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:02:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:02:59.492448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:20.686137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:20.689834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:20.768124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b947f-f24e-455a-9e00-17a3cedafe3a service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:20.773533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:20.774327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:20.774606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:20.774919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:20.783972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:20.785570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:20.795938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1157/2311] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:04:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:28.427034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.446610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8d04c-4eba-44fb-bb41-5bec566deb6e service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.450366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.451053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.451321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.451592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.453539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:04:28.474030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.475520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.487117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1155/2312] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:28.854992 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:28.864717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75001a-0e47-4d56-931a-d696d66f79ec service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:28.869104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:28.869757 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:28.870034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:28.870348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:28.882065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:28.883551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:28.895037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1158/2313] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:04:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:04:38.482999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=14 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:38.890693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=37 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:04:55.700599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:04:55.716466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557722a-671e-4462-ada6-82693272db07 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:04:55.723652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:04:55.724290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:04:55.724571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:04:55.724841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:04:55.732887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:04:55.734539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:04:55.743589 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1156/2314] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:04:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:05.739763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:15.858081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:15.872495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ab1a1-97da-4d1e-8a5c-c2f0b9689dc9 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:15.879639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:15.880378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:15.880659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:15.880928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:15.890546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:15.892202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:15.901620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1159/2315] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:05:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:24.898192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:24.909966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7870-4441-498e-b7a6-6b3ab447da36 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:24.917709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:24.917709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:24.918015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:24.918226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:24.920442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:24.933529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:24.935236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b837f2b-778c-4173-baa0-39d1fb4d755c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:24.946899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1157/2316] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:25.897209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:34.942038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:05:52.766482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:52.795456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f28b77-8622-4b95-94dd-0abcd4f15370 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:52.804536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:52.805292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:52.805636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:52.806142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:52.809713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:52.856578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:52.858505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:52.881686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1160/2317] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:54.701136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:54.701631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:54.701735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:54.728208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] MFA Rules not processed for user `175d6fedca25472796216b15ecd3bc1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:05:54.729409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:54.736464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1158/2318] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:05:55.058071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:55.067836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114135d-5ff6-47dd-97c3-15bac1503c66 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:55.071493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:55.072129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:55.072409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:55.072683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:55.074603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:55.084885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:55.086329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:55.094982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1161/2319] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:05:55.444657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:05:55.458371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1155f-e08a-450f-a9c0-b651d7acb40a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:05:55.464402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:05:55.465057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:05:55.465332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:05:55.465880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:05:55.467985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:05:55.480021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:05:55.481885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:05:55.494422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1159/2320] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:04.968884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:04.985880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7796be-2a5d-4e66-bb57-601136be6620 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:04.991365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:04.992035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:04.992318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:04.992946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:04.994894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:05.011127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:05.011127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333f4b9-645a-4728-b8a9-29e58a1e6c67 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:05.023001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1162/2321] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:05.489502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:10.932834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:06:10.945132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94f864-4f65-4c0f-a135-2cd802da6e1e service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:06:10.951249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:06:10.951906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:10.952189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:10.952466 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:10.954939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:10.968314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:06:10.969905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:06:10.994241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1160/2322] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:06:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:06:15.015906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:20.976919 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:06:25.947618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:06:25.948109 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:06:25.948235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:06:25.985317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] MFA Rules not processed for user `2401371a775743c29c16406cb9df5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:06:25.986512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:06:26.004659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1163/2323] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:06:36.002336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=14 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:21.772703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:21.786028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e472ee-b42a-4a53-b6f9-996aa64faead service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:21.793245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:21.793869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:21.794148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:21.794414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:21.803658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:21.805282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:21.814514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1161/2324] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:31.810269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:35.387587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:35.412965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027e637-12e0-4e84-b428-6d9524af77d8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:35.417366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:35.418027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:35.418365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:35.418639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:35.427030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:35.428472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:35.436729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1164/2325] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:07:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:07:45.433035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=10 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:07:56.096236 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:07:56.098189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:07:56.199245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed64c8-288a-4844-b652-245dc2ec2452 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:07:56.205366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:07:56.206030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:07:56.206295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:07:56.206556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:07:56.215274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:07:56.216933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:07:56.226527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1162/2326] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:07:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:08:06.221977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=43 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:24.731319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:24.733353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:24.812701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca2079-5b9d-4aa4-99a8-dd0558c0655a service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:24.818929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:24.819627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:24.819904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:24.820166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:24.822487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:24.835253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:24.836881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:24.847060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1165/2327] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:09:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:28.293751 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:28.295465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:28.371243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbda6d-516b-4bf5-999c-431dad3a4394 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:28.376944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:28.377835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:28.378184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:28.378526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:28.391689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:28.393128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:28.404477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1163/2328] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:09:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:29.526130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:09:29.528718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:29.544446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8053f90-1de9-404d-bcf7-d18933afeea5 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:09:29.549029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:09:29.549682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:09:29.549958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:09:29.550224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:09:29.552319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:09:29.566057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:09:29.567546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:09:29.579026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1166/2329] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:09:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:09:38.399301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=31 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:09:39.573709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=38 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:20.647168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:20.648898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:20.685731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027905ca-174a-4375-b64c-602ec310e834 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:20.689920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:20.690583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:20.690898 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:20.691162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:20.693303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:20.776829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:20.778263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c043fce-993d-48c3-86a7-4bf1af0c3383 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:20.799902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1164/2330] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:20.890686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:10:20.900627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552690a0-5380-482a-9d6d-993c0d0aa6ad service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:10:20.904892 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:10:20.905555 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:10:20.905879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:10:20.906131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:10:20.908278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:10:20.992926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:10:20.994379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:10:21.013212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1167/2331] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:10:30.794793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=50 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:10:31.010585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=41 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:25.348615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:25.349199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:25.349199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:25.392897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] MFA Rules not processed for user `2d2256afe4094555b206050fb1e09365`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:11:25.393252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:25.394579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:25.396301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:25.409377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098049-50a9-424d-9c01-370425824172 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:25.413861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:25.414515 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:25.414819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:25.415101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:25.452955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1165/2332] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:11:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:11:25.477618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:25.479208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:25.490905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1168/2333] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:11:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:25.764552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:25.775011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adb389-1628-4a48-89aa-7162d0412b62 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:25.779440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:25.780091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:25.780373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:25.780649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:25.788729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:25.790936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:25.800360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1166/2334] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:11:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:26.121270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:26.133027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366052ca-6e49-49b2-b704-9ecfd806f378 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:26.138273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:26.138974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:26.139253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:26.139521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:26.148766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:26.150459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:26.163156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1169/2335] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:11:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:29.140530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:29.163958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7750cdc-28af-4163-ba46-f04986013c44 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:29.176243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:29.177269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:29.177548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:29.178034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:29.189823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:29.191658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:29.203747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1167/2336] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:11:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:34.722375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:34.736366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b8338-da0b-45d6-a6ad-6854920146e3 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:34.743069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:34.743730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:34.744016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:34.744287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:34.754207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:34.756064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:34.767041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1170/2337] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:11:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:38.833360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:11:38.835093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:11:38.849671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4e523-72a1-49eb-9ae8-bf9b2fac2e39 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:11:38.855894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:11:38.856648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:11:38.856923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:11:38.857191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:11:38.866098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:11:38.867810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:11:38.878545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1168/2338] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:11:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:11:44.762770 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=40 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:11:48.873942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=51 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:40.328279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:40.341962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb63676-6277-44ff-900e-b4a1153a19a1 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:40.346600 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:40.347281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:40.347569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:40.347837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:40.349967 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:40.360445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:40.361939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:40.370057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1171/2339] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:12:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:12:50.366546 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:12:53.799221 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:12:53.828218 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8b2c-9335-41ea-bab6-50e46f2c393a service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:12:53.834530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:12:53.835219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:12:53.835496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:12:53.835774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:12:53.837776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:12:53.867076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:12:53.869476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:12:53.892119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1169/2340] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:12:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:13:03.889305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=20 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:14:21.597713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:21.660840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b0bab-8fda-4512-9cda-5ee6cfd1af34 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:21.665557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:21.666210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:21.666479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:21.666782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:21.669019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:21.752458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:21.753952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24ad1b-9e6d-4046-b784-4d304d9b9fd3 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:21.763720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1172/2341] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:24.994172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:25.040998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a76a72-20bd-4214-aff7-1131fedfc6aa service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:25.053258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:25.054233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:25.054939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:25.055141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:25.065712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:25.067503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:25.083100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1170/2342] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:28.164884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:28.165393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:28.165393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:28.184741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] MFA Rules not processed for user `2401371a775743c29c16406cb9df5d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:14:28.185959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:28.194875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1173/2343] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:14:28.537827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:28.563941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee074af1-5fa5-47c1-a373-406e67c1b863 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:28.569812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:28.570579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:28.570896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:28.571176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:28.573464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:28.586346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:28.587872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:28.597586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1171/2344] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:29.174316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:29.187424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e5c1d-38e4-45fa-bba9-d0a25249c9eb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:29.191369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:29.192029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:29.192336 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:29.192597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:29.194518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:14:29.221280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:29.222775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:29.235082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1174/2345] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:14:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:30.343609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:14:30.354642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2f5b1-8319-476d-b0a6-b3159610a7ef service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:14:30.359191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:14:30.359970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:14:30.360304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:14:30.360912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:14:30.371997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:14:30.373434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:14:30.384780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1172/2346] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:14:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:14:39.230055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=75 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:14:40.380024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=30 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:15:25.605053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:15:25.615455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be924d-2436-4e70-817f-37cf73b1754b service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:15:25.620191 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:15:25.620838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:15:25.621119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:15:25.621415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:15:25.630418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:15:25.631907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:15:25.640781 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1175/2347] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:15:35.636979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:16:22.306424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:16:22.316327 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fa8f1-df13-45c0-8d6c-b108f563f0e4 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:16:22.320895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:16:22.321529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:16:22.321802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:16:22.322066 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:16:22.331469 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:16:22.332885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:16:22.342458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1173/2348] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:16:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:16:32.338040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:17:01.849205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:17:01.866363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000e908-73ce-416e-a833-f2af2793ed60 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:17:01.873420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:17:01.874349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:17:01.874632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:17:01.875283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:17:01.877723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:17:01.892450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:17:01.894284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:17:01.909462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1176/2349] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:17:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:17:11.903490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:17:25.958020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:17:25.968432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418be907-aa88-4a70-8998-6976eed27ea2 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:17:25.972793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:17:25.973439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:17:25.973717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:17:25.973995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:17:25.976097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:17:26.012244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:17:26.013820 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:17:26.032235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1174/2350] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:17:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:17:36.030210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=19 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:21.821398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:21.831850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18492073-fe05-48b8-aa43-c69e24f65e66 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:21.836120 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:21.837536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:21.837977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:21.838339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:21.849797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:21.851841 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:21.863378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1177/2351] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:23.689484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:23.712220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade651-e727-4417-8801-10c87dcd5e04 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:23.722361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:23.722361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:23.722361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:23.722361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:23.726241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:23.841749 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:23.843325 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:23.868083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1175/2352] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:24.928618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:24.939091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c260488-970d-4466-a389-7bf11a18966a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:24.946973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:24.947728 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:24.948081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:24.948408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:24.950832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:24.973954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:24.975583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:24.990504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1178/2353] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:26.184659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:26.330274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134f81-39ca-4784-92f5-af9a1c8a1cda service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:26.340634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:26.341477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:26.342151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:26.342540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:26.345173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:26.386789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:26.389060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:26.403184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1176/2354] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:29.531083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:29.542194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0e2c6-0934-4568-adb1-a8d749739bd5 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:29.551845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:29.552613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:29.552893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:29.553169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:29.555356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:29.575998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:29.577950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:29.609234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1179/2355] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:30.079572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:30.095588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba184a-3082-423a-8f50-6a026a906942 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:30.103230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:30.104356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:30.106177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:30.107246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:30.110063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:30.160547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:30.165062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:30.170428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:30.179087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1177/2356] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:30.311807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:30.324882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f6398-9f55-4706-8550-605a73822473 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:30.326537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:30.328172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:30.332128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:30.332960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:30.333327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:30.333768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:30.335717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:30.346863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67e13d-16fb-4410-8318-ebd1f7703c4b service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:30.349251 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:30.350988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:30.353147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:30.354400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:30.356315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:30.356889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] PATH_INFO: `/v3/users/2d2256afe4094555b206050fb1e09365` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:30.359262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2256afe4094555b206050fb1e09365)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:30.361714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:30.362777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1180/2357] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:30.443447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:30.443447 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:30.461386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2256afe4094555b206050fb1e09365'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.462258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2256afe4094555b206050fb1e09365'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.465659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef56d3-27c4-48a6-b38d-2e0eecc8bc9a service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:30.471513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2256afe4094555b206050fb1e09365'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.471886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:30.473114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:30.473379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2256afe4094555b206050fb1e09365'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.473832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:30.474305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] PATH_INFO: `/v3/users/d43b4b2c5433485bbf8457eeea4e97b3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:30.476118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43b4b2c5433485bbf8457eeea4e97b3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:30.480761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:30.505983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2256afe4094555b206050fb1e09365'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.535063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1178/2358] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:30 2022] DELETE /identity/v3/users/2d2256afe4094555b206050fb1e09365 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:30.561966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43b4b2c5433485bbf8457eeea4e97b3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.563544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43b4b2c5433485bbf8457eeea4e97b3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.564685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43b4b2c5433485bbf8457eeea4e97b3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.582059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43b4b2c5433485bbf8457eeea4e97b3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.603737 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43b4b2c5433485bbf8457eeea4e97b3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:30.630478 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1181/2359] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:30 2022] DELETE /identity/v3/users/d43b4b2c5433485bbf8457eeea4e97b3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:30.875038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:30.876546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:30.957220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:30.959005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:31.004838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e65d-5ced-4599-97b6-94210baf9ff8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.081538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.083212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.083689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.084123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] PATH_INFO: `/v3/projects/7f112e14f33a4cd0af6e07461d8fdb4a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.085747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f112e14f33a4cd0af6e07461d8fdb4a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.088368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:31.100449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a928573-da5f-4f30-bf28-95177adf09fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.108293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.109405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.109951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.110436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] PATH_INFO: `/v3/projects/f0ac9e06d91046f7b46405794ea1203f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.121527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:31.126803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ac9e06d91046f7b46405794ea1203f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.129298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:31.176519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f112e14f33a4cd0af6e07461d8fdb4a', 'request_id': 'req-634454a3-042d-4757-b2bf-b675b2243a38'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.192824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:31.198268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f112e14f33a4cd0af6e07461d8fdb4a', 'request_id': 'req-634454a3-042d-4757-b2bf-b675b2243a38'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.216835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f112e14f33a4cd0af6e07461d8fdb4a', 'request_id': 'req-634454a3-042d-4757-b2bf-b675b2243a38'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.237363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ac9e06d91046f7b46405794ea1203f', 'request_id': 'req-e0213386-3010-441c-a73c-d469f0e02509'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.246569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ac9e06d91046f7b46405794ea1203f', 'request_id': 'req-e0213386-3010-441c-a73c-d469f0e02509'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.255475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] The token cache is being invalidate because project 7f112e14f33a4cd0af6e07461d8fdb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:31.255759 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ac9e06d91046f7b46405794ea1203f', 'request_id': 'req-e0213386-3010-441c-a73c-d469f0e02509'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.255885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.256854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-634454a3-042d-4757-b2bf-b675b2243a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.263571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1179/2360] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:30 2022] DELETE /identity/v3/projects/7f112e14f33a4cd0af6e07461d8fdb4a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:31.266137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.266645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.266952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.301751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] The token cache is being invalidate because project f0ac9e06d91046f7b46405794ea1203f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:31.303677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.306327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e0213386-3010-441c-a73c-d469f0e02509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.324777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1182/2361] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:30 2022] DELETE /identity/v3/projects/f0ac9e06d91046f7b46405794ea1203f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:31.350471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:31.352136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:31.446787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1180/2362] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:31.453499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.453789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.454409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.544462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:31.550913 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:31.575270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1183/2363] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:31.579311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:31.594807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe433fa-0519-4fa3-96f1-0a7c50ebf226 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.610177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.610900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.611182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.611453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.612251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:31.612687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.614596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:31.660153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1181/2364] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:18:31.664686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:31.676895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873d51f-04bf-44e1-9f67-be3da87d6eec None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.682599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.683633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.684151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.684773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.685723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.688505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:31.718502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:31.720686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:31.745597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ee984-ee38-4bb0-b0b9-79be4f4597fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.759356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.760476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.760920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.761354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] PATH_INFO: `/v3/users/175d6fedca25472796216b15ecd3bc1a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.763511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] RBAC: Authorizing `identity:delete_user(user_id=175d6fedca25472796216b15ecd3bc1a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.766281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:31.783325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1184/2365] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:31 2022] POST /identity/v3/projects => generated 403 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:31.789230 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:31.837856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175d6fedca25472796216b15ecd3bc1a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.838509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175d6fedca25472796216b15ecd3bc1a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.847479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175d6fedca25472796216b15ecd3bc1a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.848070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175d6fedca25472796216b15ecd3bc1a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.850114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:31.866400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175d6fedca25472796216b15ecd3bc1a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:31.876116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dedcf7-3938-4503-a51d-669589a58c34 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.880272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1182/2366] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:31 2022] DELETE /identity/v3/users/175d6fedca25472796216b15ecd3bc1a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:31.880743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.881517 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.881798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.882077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.882772 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.885040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:31.908472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1185/2367] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:31 2022] POST /identity/v3/users => generated 302 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:31.912862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:31.914340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:31.985128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b835b-574c-422d-8192-1bc9be4d310c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:31.990125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:31.991049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:31.991407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:31.991704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:31.992505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:31.994176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.004872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1183/2368] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:32.008872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.019482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a7b08-1ed2-4a1a-b9c6-084f1e0dabbd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.024647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.025418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.025701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.025975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.026776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.028806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.039268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1186/2369] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:32.043616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.054694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae09376-42e2-4e0c-9c8d-0f6ade9a99bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.059648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61f7ee-02f4-4f63-8ce5-525192a28848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.060547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f7ee-02f4-4f63-8ce5-525192a28848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.060827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f7ee-02f4-4f63-8ce5-525192a28848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.091461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f7ee-02f4-4f63-8ce5-525192a28848 admin admin] PATH_INFO: `/v3/projects/38f1c4fa57184a2ab2f8946812d41358/users/e4643a0b498b4fe5833a8acc7a87646e/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.093448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.093448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.093448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43335f-d7ad-4069-af32-b76fd1e4dfcb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.093611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61f7ee-02f4-4f63-8ce5-525192a28848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f1c4fa57184a2ab2f8946812d41358, user_id=e4643a0b498b4fe5833a8acc7a87646e, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.093945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61f7ee-02f4-4f63-8ce5-525192a28848 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.101416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.103834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.104978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.105942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] PATH_INFO: `/v3/projects/2f93c08bb4234073b8dd996119d06701` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.109869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f93c08bb4234073b8dd996119d06701)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.115974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.116568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1184/2370] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:32 2022] PUT /identity/v3/projects/38f1c4fa57184a2ab2f8946812d41358/users/e4643a0b498b4fe5833a8acc7a87646e/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:32.127331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.127896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.128074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.157904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] MFA Rules not processed for user `e4643a0b498b4fe5833a8acc7a87646e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:32.159234 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.175094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:32.178505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f93c08bb4234073b8dd996119d06701', 'request_id': 'req-98f9e048-6765-43a2-b05a-61003f9185f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:32.186576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f93c08bb4234073b8dd996119d06701', 'request_id': 'req-98f9e048-6765-43a2-b05a-61003f9185f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:32.193349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f93c08bb4234073b8dd996119d06701', 'request_id': 'req-98f9e048-6765-43a2-b05a-61003f9185f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:32.208648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] The token cache is being invalidate because project 2f93c08bb4234073b8dd996119d06701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:32.208940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:32.209604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98f9e048-6765-43a2-b05a-61003f9185f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:32.211306 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1187/2371] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:32 2022] DELETE /identity/v3/projects/2f93c08bb4234073b8dd996119d06701 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:32.217531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0680e-dbb7-462f-aac1-51de22f6dbae None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.217794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0680e-dbb7-462f-aac1-51de22f6dbae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.218076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0680e-dbb7-462f-aac1-51de22f6dbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.231651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1185/2372] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:32.236023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.237494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.240548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4dd0680e-dbb7-462f-aac1-51de22f6dbae None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:32.241784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4dd0680e-dbb7-462f-aac1-51de22f6dbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.283871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:32.310198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1188/2373] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:32.311892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bec32d-f87a-4308-98b2-e540987fbc19 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.316804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.317478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.317757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.318043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.318413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.318697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.318795 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.319005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.320723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.342718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:32.344072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.351384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1186/2374] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/projects => generated 403 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:32.352049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1189/2375] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:32.355607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.355976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.367227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb071e90-3780-4d2e-989f-58b4bd8468cd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.421669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cacf9e-caee-49da-95d4-293051579272 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.426865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.427546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.427829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.428106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.428801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.433896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.434063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.434063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.434063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.434063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.434329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:32.434329 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.435370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.451920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1190/2376] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:18:32.452977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.453506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1187/2377] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:32.457586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.463611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3f7fe-8ad6-4622-be3b-d57a92ecf246 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.468488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca28b5-2975-4b55-a8ce-9c3f45ef1958 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.468996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.469747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.470106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.470449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.471256 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.473235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.473338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.473925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.474242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.474522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.475311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.476931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.487829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1188/2378] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:32.492311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.503880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fee5-e826-4d6d-9ee6-6d74ca492aac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.506281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1191/2379] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:32.510347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.552387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:32.564857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.565790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.566154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.566629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] PATH_INFO: `/v3/projects/5ecb960b6acd4587b25f3b19693148f1/users/da198f4ce0a246feac3888afebe784d8/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.575544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee96ec1-d5f4-4c42-834b-a89b8eab341d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.580452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.581136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.581417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.581694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.582454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.584472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.587665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ecb960b6acd4587b25f3b19693148f1, user_id=da198f4ce0a246feac3888afebe784d8, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.593298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.606994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1192/2380] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:32.612704 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.616754 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1189/2381] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:32 2022] PUT /identity/v3/projects/5ecb960b6acd4587b25f3b19693148f1/users/da198f4ce0a246feac3888afebe784d8/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:32.620426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.625267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad17ad9-e2eb-4cf5-a204-f1e40992739b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.660602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:32.701565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6926c19-c1a7-487f-a787-ba4172375165 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.709860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.710674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.710804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.711123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.711470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.711625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.711967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.712306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.712487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.713150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.714664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.715288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.725962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1193/2382] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:32.727114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.727217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1190/2383] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:32.730162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.731690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.741180 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a11504-6f2a-487e-8a62-ced50c64c56f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.746135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.746922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.747267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.747608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.748413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.750136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.760588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1191/2384] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:32.761983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.775771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25a51b-4887-4d4d-b600-ab7421fc1cdf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.782358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.783644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.784065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.784423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] PATH_INFO: `/v3/projects/5ecb960b6acd4587b25f3b19693148f1/users/da198f4ce0a246feac3888afebe784d8/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.786958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ecb960b6acd4587b25f3b19693148f1, user_id=da198f4ce0a246feac3888afebe784d8, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.789744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.802413 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47868588-799c-4895-b53e-83368e60bac0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.804141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1192/2385] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:32 2022] PUT /identity/v3/projects/5ecb960b6acd4587b25f3b19693148f1/users/da198f4ce0a246feac3888afebe784d8/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:32.805126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:32.868909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.869588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.869871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.870147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.872060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.878573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ca3b-e6fa-4835-9ebf-473f522bc867 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:32.883346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66870d6-1e52-458d-8c30-5b1d96e659bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:32.884240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66870d6-1e52-458d-8c30-5b1d96e659bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.884534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66870d6-1e52-458d-8c30-5b1d96e659bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.884845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66870d6-1e52-458d-8c30-5b1d96e659bb admin admin] PATH_INFO: `/v3/projects/d19013d24b834a0ba1565d952264b80e/users/a63be4691fe1438991c191520b9af61f/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.884942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.886374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abd0fe-83cf-4467-8fb8-05a5b91d0fb7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.896682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1194/2386] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:32.898018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.898307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.898583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.903096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66870d6-1e52-458d-8c30-5b1d96e659bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d19013d24b834a0ba1565d952264b80e, user_id=a63be4691fe1438991c191520b9af61f, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:32.907510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66870d6-1e52-458d-8c30-5b1d96e659bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:32.926350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1193/2387] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:32 2022] PUT /identity/v3/projects/d19013d24b834a0ba1565d952264b80e/users/a63be4691fe1438991c191520b9af61f/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:32.935268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] MFA Rules not processed for user `da198f4ce0a246feac3888afebe784d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:32.936556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:32.936671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:32.936850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:32.937125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:32.950391 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:32.974108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] MFA Rules not processed for user `a63be4691fe1438991c191520b9af61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:32.975576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.024437 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1195/2388] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:33.037761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.039440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.048448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1194/2389] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:33.053279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.067624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:33.111059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3af52-3094-45ba-aed2-3065ec502754 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.114958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.115624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.115898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.116166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.125980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.127481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.130502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c100dd-66b1-4b0f-b26d-20199cc587b8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.136915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.137879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.138415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.138867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.139731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.142046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.149114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1196/2390] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:33.175813 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1195/2391] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:33 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:33.179791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.196425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003d55-1c9a-42ff-b42b-43631b2488a9 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.263142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.263974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.264355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.264749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.265552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.268171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.294896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.299832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1197/2392] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:33 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:33.304076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.310372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8f609-8c1d-497f-8a87-99e03a35136e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.315647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.316572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.316858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.317130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] PATH_INFO: `/v3/users/e4643a0b498b4fe5833a8acc7a87646e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.318365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4643a0b498b4fe5833a8acc7a87646e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.319477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac08b6-c3ed-4b65-b98a-49d359174fd7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.320828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.324678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.325484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.326005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.326428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.327359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.329287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.348052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1198/2393] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:33.352902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.372152 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8a4d5-524d-483a-901f-0150db851387 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.380198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.381593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.381946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.382319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] PATH_INFO: `/v3/projects/ad663bf380e24e7db211a35742c4c0c7/users/5a4368bcd662415fb0bdd50b3d15d42a/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.399068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad663bf380e24e7db211a35742c4c0c7, user_id=5a4368bcd662415fb0bdd50b3d15d42a, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.399354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4643a0b498b4fe5833a8acc7a87646e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:33.400158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4643a0b498b4fe5833a8acc7a87646e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:33.401822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.409558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4643a0b498b4fe5833a8acc7a87646e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:33.410617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4643a0b498b4fe5833a8acc7a87646e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:33.418961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1199/2394] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:33 2022] PUT /identity/v3/projects/ad663bf380e24e7db211a35742c4c0c7/users/5a4368bcd662415fb0bdd50b3d15d42a/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:33.422922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.425983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.427429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9afe33f2-ba3a-44b8-bff3-5117f25afa35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4643a0b498b4fe5833a8acc7a87646e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:33.437642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:33.446268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1196/2395] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:33 2022] DELETE /identity/v3/users/e4643a0b498b4fe5833a8acc7a87646e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:33.450786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.450955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.451229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.471028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:33.472277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.511511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfcd09b-7e64-46f1-8bf7-1192af8b250e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.542091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1197/2396] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:33.555879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.557428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.568172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:33.575991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.576849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.577192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.577601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.578427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.580371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.591857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1200/2397] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:33.595776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.605930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75cae71-b1e3-4831-9204-2e3cd67b5ac9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.612287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781e21a-e273-4bab-abdd-423e2db95442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.613323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e21a-e273-4bab-abdd-423e2db95442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.613742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e21a-e273-4bab-abdd-423e2db95442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.614087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e21a-e273-4bab-abdd-423e2db95442 admin admin] PATH_INFO: `/v3/projects/ad663bf380e24e7db211a35742c4c0c7/users/5a4368bcd662415fb0bdd50b3d15d42a/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.616162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781e21a-e273-4bab-abdd-423e2db95442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad663bf380e24e7db211a35742c4c0c7, user_id=5a4368bcd662415fb0bdd50b3d15d42a, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.618916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781e21a-e273-4bab-abdd-423e2db95442 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.626016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbdf52-27c1-40eb-9c48-84a3198a922e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.630416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1201/2398] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:33 2022] PUT /identity/v3/projects/ad663bf380e24e7db211a35742c4c0c7/users/5a4368bcd662415fb0bdd50b3d15d42a/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:33.642033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.642361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.642633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.672994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] MFA Rules not processed for user `5a4368bcd662415fb0bdd50b3d15d42a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:33.678367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.688780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.689855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.690061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.690341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.786831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1202/2399] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:33.789960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.791504 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.798281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:33.802056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:33.807199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1198/2400] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:33.849156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:33.888902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfbe32-c7c1-466e-b710-48cd6ea96c0c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:33.893175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:33.893858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:33.894137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:33.894407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:33.907981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:33.909590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:33.931659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1203/2401] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:34.019623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:34.030442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f15c43-5ff7-46b4-9d92-5a39783b9f56 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:34.034461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:34.035176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:34.035488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:34.035765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:34.037750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:34.118881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:34.120609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:34.131720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1199/2402] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:34.492940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:34.494634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:34.513469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44229a18-53bc-49c4-9cee-46ceeb24bb30 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:34.519094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:34.520296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:34.520813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:34.521285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] PATH_INFO: `/v3/projects/38f1c4fa57184a2ab2f8946812d41358` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:34.522800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f1c4fa57184a2ab2f8946812d41358)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:34.524849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:34.572762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f1c4fa57184a2ab2f8946812d41358', 'request_id': 'req-75554f12-5b72-488b-9a5f-96bba167fb71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.579265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f1c4fa57184a2ab2f8946812d41358', 'request_id': 'req-75554f12-5b72-488b-9a5f-96bba167fb71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.587616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f1c4fa57184a2ab2f8946812d41358', 'request_id': 'req-75554f12-5b72-488b-9a5f-96bba167fb71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.603197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] The token cache is being invalidate because project 38f1c4fa57184a2ab2f8946812d41358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:34.603638 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.604313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.606364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1204/2403] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:34 2022] DELETE /identity/v3/projects/38f1c4fa57184a2ab2f8946812d41358 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:34.610519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:34.621803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ad7a6-0bd8-46b9-9180-c659749810c7 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:34.629757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:34.682141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:34.682966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:34.683241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:34.683507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] PATH_INFO: `/v3/users/da198f4ce0a246feac3888afebe784d8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:34.684806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da198f4ce0a246feac3888afebe784d8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:34.686717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:34.745556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da198f4ce0a246feac3888afebe784d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.746244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da198f4ce0a246feac3888afebe784d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.754442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da198f4ce0a246feac3888afebe784d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.755305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da198f4ce0a246feac3888afebe784d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.787377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a77040de-e857-4179-aa10-5911c0a05299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da198f4ce0a246feac3888afebe784d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:34.820472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1200/2404] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:34 2022] DELETE /identity/v3/users/da198f4ce0a246feac3888afebe784d8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:35.125741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.127276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:35.141773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75554f12-5b72-488b-9a5f-96bba167fb71 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.212995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.214243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.214579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.214946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] PATH_INFO: `/v3/projects/5ecb960b6acd4587b25f3b19693148f1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.216203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ecb960b6acd4587b25f3b19693148f1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.218322 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.279489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ecb960b6acd4587b25f3b19693148f1', 'request_id': 'req-11d08ef6-deef-42bf-903f-ec9e3b911525'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:35.280957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:35.288279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ecb960b6acd4587b25f3b19693148f1', 'request_id': 'req-11d08ef6-deef-42bf-903f-ec9e3b911525'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:35.297509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ecb960b6acd4587b25f3b19693148f1', 'request_id': 'req-11d08ef6-deef-42bf-903f-ec9e3b911525'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:35.313799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] The token cache is being invalidate because project 5ecb960b6acd4587b25f3b19693148f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:35.314089 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:35.314711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d08ef6-deef-42bf-903f-ec9e3b911525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:35.316495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1205/2405] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:35 2022] DELETE /identity/v3/projects/5ecb960b6acd4587b25f3b19693148f1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:35.321581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.322108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.322108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.340963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:35.342175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:35.412113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1201/2406] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:35.419798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.420088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.420365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.442631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:35.444933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:35.455202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1206/2407] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:35.460646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.472895 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de3dec-d1e1-4125-ad0f-e312bbd8aabf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.477629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.478301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.478579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.478907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.479676 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:35.480007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.481610 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.494121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1202/2408] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:18:35.498311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.508971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3cb63-6db3-4770-8967-ab2624ef7a06 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.513859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.514564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.514881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.515159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.515898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.517831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.549147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1207/2409] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:35 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:35.553802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.565801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea13d8-8042-4577-80b3-2ae0020c0910 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.589480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:35.663924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.664822 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.665279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.665716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.666642 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.668878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.694269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1203/2410] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:35 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:35.698031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.711525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbadbc-c041-40b4-a8c7-eb0680826acb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.723086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.724522 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.724897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.725326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.726363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.727866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.749484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1208/2411] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:35.756545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.772501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d9caf-c149-4b5b-a28e-76e110705619 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.790922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b03fbe-c7ea-422c-bfeb-7b0ebb7b7fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.791624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b03fbe-c7ea-422c-bfeb-7b0ebb7b7fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.791899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b03fbe-c7ea-422c-bfeb-7b0ebb7b7fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.792167 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b03fbe-c7ea-422c-bfeb-7b0ebb7b7fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.792955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b03fbe-c7ea-422c-bfeb-7b0ebb7b7fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.795084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b03fbe-c7ea-422c-bfeb-7b0ebb7b7fdf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.808420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1204/2412] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:35.815022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:35.831628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086b084-a5ff-4476-a3dc-3b030f8eef72 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:35.853010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:35.854323 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.854608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.855085 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] PATH_INFO: `/v3/projects/7be1a142fcc74c0091165367b122cae0/users/1cac05c9c0c24c249f65240da0c5434f/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.888268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be1a142fcc74c0091165367b122cae0, user_id=1cac05c9c0c24c249f65240da0c5434f, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:35.891121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:35.909069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1209/2413] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:35 2022] PUT /identity/v3/projects/7be1a142fcc74c0091165367b122cae0/users/1cac05c9c0c24c249f65240da0c5434f/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:35.921517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:35.921810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:35.922123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:35.953767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] MFA Rules not processed for user `1cac05c9c0c24c249f65240da0c5434f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:35.955276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:36.045981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1205/2414] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:36.048991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.155989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f018ddc-159d-4471-a4b6-64a4a0c5fa83 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.162165 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.162939 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.163260 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.163536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.164296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.166932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.216788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1210/2415] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:36 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:36.221332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.239336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde6546-5d74-4600-9c3e-10cd0944f2e2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.325491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.326182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.326698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.326830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.327453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.329391 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.356591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1206/2416] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:36 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:36.360267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.371702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b78d6-4784-4f37-b72c-3071d3d04c3c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.384699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.385406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.385688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.385965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.386717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.389390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.411273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1211/2417] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:36.412658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.427222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf798c8-031e-41f5-890e-79a9a53c3100 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.432720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.433932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.434288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.434662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] PATH_INFO: `/v3/projects/06b3f5216b00470e8dd8433c59db39e0/users/b9aa1b465efa4dd7b76a4d2a64c3f7ff/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.449928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:36.454282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b3f5216b00470e8dd8433c59db39e0, user_id=b9aa1b465efa4dd7b76a4d2a64c3f7ff, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.459557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.479222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1207/2418] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:36 2022] PUT /identity/v3/projects/06b3f5216b00470e8dd8433c59db39e0/users/b9aa1b465efa4dd7b76a4d2a64c3f7ff/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:36.485005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.498330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279b79f-68e6-46d3-b149-b011bc45fd27 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.579459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:36.594977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.595684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.595965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.596237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.596986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.598665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.609936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1212/2419] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:36.614437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.626223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1af09a-4d14-4921-a018-29d20ef70b10 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.634402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b26eb8-2173-4e40-be83-8c67552568ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.635306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b26eb8-2173-4e40-be83-8c67552568ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.635619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b26eb8-2173-4e40-be83-8c67552568ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.635843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b26eb8-2173-4e40-be83-8c67552568ee admin admin] PATH_INFO: `/v3/projects/06b3f5216b00470e8dd8433c59db39e0/users/b9aa1b465efa4dd7b76a4d2a64c3f7ff/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.637816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b26eb8-2173-4e40-be83-8c67552568ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b3f5216b00470e8dd8433c59db39e0, user_id=b9aa1b465efa4dd7b76a4d2a64c3f7ff, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.639627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.640516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b26eb8-2173-4e40-be83-8c67552568ee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.642218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:36.655910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1208/2420] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:36 2022] PUT /identity/v3/projects/06b3f5216b00470e8dd8433c59db39e0/users/b9aa1b465efa4dd7b76a4d2a64c3f7ff/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:36.659246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b275a7-8ce7-4b29-8734-f7c2ebcc178b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.669388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.669669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.669944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.700039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] MFA Rules not processed for user `b9aa1b465efa4dd7b76a4d2a64c3f7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:36.703292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:36.731162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.732050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.732330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.732605 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] PATH_INFO: `/v3/users/73d50022c5f94444afc67ea810e4d6fe` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.733820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d50022c5f94444afc67ea810e4d6fe)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:36.735865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:36.778414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1209/2421] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:36.788685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d50022c5f94444afc67ea810e4d6fe'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:36.789365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d50022c5f94444afc67ea810e4d6fe'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:36.790048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d50022c5f94444afc67ea810e4d6fe'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:36.790936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:36.792519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:36.809280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d50022c5f94444afc67ea810e4d6fe'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:36.816527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d50022c5f94444afc67ea810e4d6fe'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:36.829801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1213/2422] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:36 2022] DELETE /identity/v3/users/73d50022c5f94444afc67ea810e4d6fe => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:36.862276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cab3d-28d9-4577-8e9d-1fa471c5dcad None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:36.916415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:36.917200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:36.917383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:36.917643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:36.919539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:36.963210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:36.998153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.000554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.014973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1210/2423] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:37.044429 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.045983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:37.128719 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.130507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:37.136072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533d393-0309-44bd-b2d4-22bff8a735b3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.141784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.142792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.143111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.143418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] PATH_INFO: `/v3/projects/72b4ce05423043b69e280264439ddbe6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.144826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b328b49-c626-43a3-b0d7-286a99e840da service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.144934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b4ce05423043b69e280264439ddbe6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.146916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.148767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.149600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.149931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.150196 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.152210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:37.155460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:37.207929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b4ce05423043b69e280264439ddbe6', 'request_id': 'req-4a8178c2-4552-4f52-9b87-53d846bcb7a9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:37.214777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b4ce05423043b69e280264439ddbe6', 'request_id': 'req-4a8178c2-4552-4f52-9b87-53d846bcb7a9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:37.220938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b4ce05423043b69e280264439ddbe6', 'request_id': 'req-4a8178c2-4552-4f52-9b87-53d846bcb7a9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:37.226845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.228556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479077e-6374-4714-84cd-b622fcb435d9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.239106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] The token cache is being invalidate because project 72b4ce05423043b69e280264439ddbe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:37.239398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:37.239613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1211/2424] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:37.240156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4a8178c2-4552-4f52-9b87-53d846bcb7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:37.241743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1214/2425] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:37 2022] DELETE /identity/v3/projects/72b4ce05423043b69e280264439ddbe6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:37.248528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.248855 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.249179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.269562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:37.270771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:37.344481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1212/2426] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:37.354378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.354909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.355307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.378635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:37.380353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:37.398870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1215/2427] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:37.403124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.414728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9ea49-5205-4676-8d4b-248779169b9b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.419565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.420225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.420506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.420778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.421536 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:37.421837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.423494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.438789 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1213/2428] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:18:37.441674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.451164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce8be-0eca-41ce-8cd4-a555ab104786 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.455838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.456505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.456783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.457096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.458138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.459802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.494857 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1216/2429] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:37.496941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.508416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f615a6-614d-4c1b-a80e-1191ca9c03a4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.559932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:37.567978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.568656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.568936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.569260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.569985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.571974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.597289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1214/2430] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:37.598389 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.609572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfcb0c-c9eb-4034-8715-224bb995458f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.614570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.615282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.615578 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.615839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.616591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.618237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.629219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1217/2431] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:37.633351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.644237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0554b-d5ac-49e6-8694-fa0bf5c470ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.649034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d711c8-c174-4079-9efb-a99028f36410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.649733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d711c8-c174-4079-9efb-a99028f36410 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.650016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d711c8-c174-4079-9efb-a99028f36410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.650284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d711c8-c174-4079-9efb-a99028f36410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.651042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d711c8-c174-4079-9efb-a99028f36410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.652614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d711c8-c174-4079-9efb-a99028f36410 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.663276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1215/2432] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:37.667649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.678065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457797e-e0a8-4d22-b948-22ead80ce90d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.683015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.683917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.684199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.684474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] PATH_INFO: `/v3/projects/a6178c6ccad6448a9a676bdd67a34c68/users/7e36a64959eb419fbf0e3059ebcec772/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.699227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6178c6ccad6448a9a676bdd67a34c68, user_id=7e36a64959eb419fbf0e3059ebcec772, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.701929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.720692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1218/2433] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:37 2022] PUT /identity/v3/projects/a6178c6ccad6448a9a676bdd67a34c68/users/7e36a64959eb419fbf0e3059ebcec772/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:37.731819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.732115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.732392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.761081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] MFA Rules not processed for user `7e36a64959eb419fbf0e3059ebcec772`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:37.762260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:37.836614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1216/2434] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:37.841483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.906307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d328d-b7c6-45b6-871d-6eaffcbd92c4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:37.911769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:37.912446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:37.912756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:37.913026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:37.913717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:37.915687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:37.946385 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1219/2435] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:37.947641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:37.959613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33601b-48f3-4d3c-b13b-ee795b0d63d5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:38.017771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:38.018368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.018645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.018951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.019654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:38.021572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:38.042320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1217/2436] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:37 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:38.046769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:38.058695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720dccc-164d-4105-b04e-41970a5dcb7a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:38.064335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:38.065006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.065282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.065553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.066291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:38.068026 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:38.079202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1220/2437] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:38.083797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:38.095034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021ffa1-9543-4f77-930a-1797296a88e6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:38.100940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:38.101824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.102107 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.102385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] PATH_INFO: `/v3/projects/370f77461c6d4f9c86b8927817ca1cef/users/08964e34faea4cdfa4f39480bc6af564/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.118056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=370f77461c6d4f9c86b8927817ca1cef, user_id=08964e34faea4cdfa4f39480bc6af564, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:38.120800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:38.135819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1218/2438] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:38 2022] PUT /identity/v3/projects/370f77461c6d4f9c86b8927817ca1cef/users/08964e34faea4cdfa4f39480bc6af564/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:38.140442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:38.149937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:38.155302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8781-b5f2-4e96-a8df-e57e4cdf82d6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:38.214858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b45ea-11bb-4b4f-8a4f-25d58d4c869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:38.215516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b45ea-11bb-4b4f-8a4f-25d58d4c869e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.215797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b45ea-11bb-4b4f-8a4f-25d58d4c869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.216062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b45ea-11bb-4b4f-8a4f-25d58d4c869e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.216777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b45ea-11bb-4b4f-8a4f-25d58d4c869e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:38.218371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b45ea-11bb-4b4f-8a4f-25d58d4c869e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:38.228803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1221/2439] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:38.233881 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:38.250446 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a0690-c8ec-4e9e-a29f-5ba6cf12df6b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:38.257843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:38.258728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.259051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.259298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] PATH_INFO: `/v3/projects/370f77461c6d4f9c86b8927817ca1cef/users/08964e34faea4cdfa4f39480bc6af564/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.261603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=370f77461c6d4f9c86b8927817ca1cef, user_id=08964e34faea4cdfa4f39480bc6af564, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:38.264359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:38.275499 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1219/2440] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:38 2022] PUT /identity/v3/projects/370f77461c6d4f9c86b8927817ca1cef/users/08964e34faea4cdfa4f39480bc6af564/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:38.286233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.286588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.286983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.319777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] MFA Rules not processed for user `08964e34faea4cdfa4f39480bc6af564`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:38.321514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:38.403824 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1222/2441] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:38.415150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:38.417269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:38.438074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:38.479021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33835aea-6547-4e7a-8208-3a6769414120 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:38.483652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c5a93-9954-4b7b-bdbe-97795f676a90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:38.484520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c5a93-9954-4b7b-bdbe-97795f676a90 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:38.484852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c5a93-9954-4b7b-bdbe-97795f676a90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:38.485192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c5a93-9954-4b7b-bdbe-97795f676a90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:38.497458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c5a93-9954-4b7b-bdbe-97795f676a90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:38.499374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c5a93-9954-4b7b-bdbe-97795f676a90 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:38.511363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1220/2442] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:40.021815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:40.089725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a03d-1c05-4ad8-b594-42fc62dd0988 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:40.094514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:40.095480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:40.095766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:40.096043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] PATH_INFO: `/v3/users/7e36a64959eb419fbf0e3059ebcec772` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:40.097298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e36a64959eb419fbf0e3059ebcec772)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:40.099317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:40.139670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:40.165218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e36a64959eb419fbf0e3059ebcec772'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.166350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e36a64959eb419fbf0e3059ebcec772'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.167270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e36a64959eb419fbf0e3059ebcec772'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.185093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e36a64959eb419fbf0e3059ebcec772'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.193474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e36a64959eb419fbf0e3059ebcec772'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.209088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1223/2443] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:40 2022] DELETE /identity/v3/users/7e36a64959eb419fbf0e3059ebcec772 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:40.214836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:40.215322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:40.215407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:40.234514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:40.235831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:40.309164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1221/2444] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:40.322039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:40.323619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:40.388511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126340d-c04d-4bcd-a165-db8db801bf0e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:40.392869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:40.393509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:40.393785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:40.394020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:40.405331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:40.406880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:40.417261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1224/2445] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:40.718432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:40.720121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:40.734060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aba5d6-0a66-40d7-8c0c-e71785059314 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:40.739239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:40.740185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:40.740459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:40.740733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] PATH_INFO: `/v3/projects/a6178c6ccad6448a9a676bdd67a34c68` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:40.742059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6178c6ccad6448a9a676bdd67a34c68)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:40.744079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:40.795581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6178c6ccad6448a9a676bdd67a34c68', 'request_id': 'req-b5c94e3a-65ea-431c-a676-00dcd24db29d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.802639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6178c6ccad6448a9a676bdd67a34c68', 'request_id': 'req-b5c94e3a-65ea-431c-a676-00dcd24db29d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.808947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6178c6ccad6448a9a676bdd67a34c68', 'request_id': 'req-b5c94e3a-65ea-431c-a676-00dcd24db29d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.823514 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] The token cache is being invalidate because project a6178c6ccad6448a9a676bdd67a34c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:40.823739 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.824375 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.825957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1222/2446] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:40 2022] DELETE /identity/v3/projects/a6178c6ccad6448a9a676bdd67a34c68 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:40.829995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:40.840272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041dcb3-6338-4239-bfd5-df4ba5f75f78 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:40.902820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:40.903751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:40.904029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:40.904311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] PATH_INFO: `/v3/users/08964e34faea4cdfa4f39480bc6af564` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:40.905683 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08964e34faea4cdfa4f39480bc6af564)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:40.907755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:40.961373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08964e34faea4cdfa4f39480bc6af564'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.962043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08964e34faea4cdfa4f39480bc6af564'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.962680 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08964e34faea4cdfa4f39480bc6af564'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.965479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:40.978736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08964e34faea4cdfa4f39480bc6af564'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:40.986254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b895a3be-83da-4895-b97b-dc044410264f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08964e34faea4cdfa4f39480bc6af564'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:41.000684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1225/2447] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:40 2022] DELETE /identity/v3/users/08964e34faea4cdfa4f39480bc6af564 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:41.293340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.294966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:41.308173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c94e3a-65ea-431c-a676-00dcd24db29d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.350277 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:41.372219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.373098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.373377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.373649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] PATH_INFO: `/v3/projects/370f77461c6d4f9c86b8927817ca1cef` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.375039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=370f77461c6d4f9c86b8927817ca1cef)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.377013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.431189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370f77461c6d4f9c86b8927817ca1cef', 'request_id': 'req-48508bd5-9ca9-4548-b46f-430d9b20f4ee'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:41.445118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370f77461c6d4f9c86b8927817ca1cef', 'request_id': 'req-48508bd5-9ca9-4548-b46f-430d9b20f4ee'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:41.455130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370f77461c6d4f9c86b8927817ca1cef', 'request_id': 'req-48508bd5-9ca9-4548-b46f-430d9b20f4ee'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:41.475836 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] The token cache is being invalidate because project 370f77461c6d4f9c86b8927817ca1cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:41.476215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:41.477227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48508bd5-9ca9-4548-b46f-430d9b20f4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:41.479770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1223/2448] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:41 2022] DELETE /identity/v3/projects/370f77461c6d4f9c86b8927817ca1cef => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:41.487326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.487816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.487921 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.508046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:41.509268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:41.582708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1226/2449] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:41.591539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.591834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.592112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.613707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:41.615106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:41.630432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1224/2450] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:41.637123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.650370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1bd6-93b4-46ce-94be-67b99417331b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.656612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.657270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.657554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.657812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.658569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:41.658904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.660548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.673044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1227/2451] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:18:41.678403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.692897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885526a7-b943-48b8-85b9-5eae6cf1c0c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.699935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.700583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.700862 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.701126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.701808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.703816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.733526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1225/2452] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:41 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:41.737952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.748471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086975a-822b-46bf-bff3-91a88e00b001 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.809034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.809739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.810020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.810350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.811061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.813047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.835279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1228/2453] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:41 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:41.839956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.850819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c02bc-0d4f-45fb-9813-d9e4fce99a61 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.855726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.856380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.856659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.856928 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.857674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.859295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.869639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1226/2454] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:41.875179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.885946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a8ca8-4a5a-4618-89f9-906913975c3e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.891969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191d1a0-94a6-44ee-8e46-11f10c5b2305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.892665 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191d1a0-94a6-44ee-8e46-11f10c5b2305 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.892943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191d1a0-94a6-44ee-8e46-11f10c5b2305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.893213 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191d1a0-94a6-44ee-8e46-11f10c5b2305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.893956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191d1a0-94a6-44ee-8e46-11f10c5b2305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.895621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191d1a0-94a6-44ee-8e46-11f10c5b2305 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.906100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1229/2455] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:41.910496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:41.920775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4348a5-0ea8-4693-a118-54177777ebd5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:41.925938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:41.926799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.927099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.927484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] PATH_INFO: `/v3/projects/f76e9a61af6346f3b0e17f84c3599634/users/e8b359217aa14103b5d6aea07a525d05/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:41.942365 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76e9a61af6346f3b0e17f84c3599634, user_id=e8b359217aa14103b5d6aea07a525d05, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:41.945155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:41.959731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1227/2456] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:41 2022] PUT /identity/v3/projects/f76e9a61af6346f3b0e17f84c3599634/users/e8b359217aa14103b5d6aea07a525d05/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:41.970534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:41.970957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:41.971404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.000786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] MFA Rules not processed for user `e8b359217aa14103b5d6aea07a525d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:42.002035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:42.024822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:42.073093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1230/2457] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:42.077341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.138388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbed4d3-345b-4256-b277-81966b0f401b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.143057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.143707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.143982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.144270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.144942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.146901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.162650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:42.181200 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1228/2458] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:42 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:42.182370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.193423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd18b86-292c-46a5-9d00-92e8c25694c8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.252268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.252944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.253226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.253501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.254198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.256479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.278150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1231/2459] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:42 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:42.282574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.293456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b74e-3e64-41f8-b394-7ca38ed1ee9e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.299114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.300611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.300947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.301254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.302119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.304405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.315723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1229/2460] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:42.320220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.331229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606898f-ba5b-4421-ac99-1d99cd82a144 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.337018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.337940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.338116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.338668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] PATH_INFO: `/v3/projects/01eed35e3c0a4e8d80b65adb069df945/users/47d211e5501f4685bc01e29847e6d498/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.356240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01eed35e3c0a4e8d80b65adb069df945, user_id=47d211e5501f4685bc01e29847e6d498, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.359069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.373978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1232/2461] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:42 2022] PUT /identity/v3/projects/01eed35e3c0a4e8d80b65adb069df945/users/47d211e5501f4685bc01e29847e6d498/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:42.378060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.388821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d6ac0-f76c-4b52-97f4-73daaeecd0ae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.451157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254cf6a-1105-49dd-8c8d-0152582b9f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.451930 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254cf6a-1105-49dd-8c8d-0152582b9f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.452272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254cf6a-1105-49dd-8c8d-0152582b9f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.452564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254cf6a-1105-49dd-8c8d-0152582b9f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.453708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254cf6a-1105-49dd-8c8d-0152582b9f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.455534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254cf6a-1105-49dd-8c8d-0152582b9f6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.468067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1230/2462] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:42.472547 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.482883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06b1cb-4ca5-4eb0-8382-f4797d75b0a3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.487500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.488384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.488663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.488935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] PATH_INFO: `/v3/projects/01eed35e3c0a4e8d80b65adb069df945/users/47d211e5501f4685bc01e29847e6d498/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.491114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01eed35e3c0a4e8d80b65adb069df945, user_id=47d211e5501f4685bc01e29847e6d498, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.493858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.505530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1233/2463] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:42 2022] PUT /identity/v3/projects/01eed35e3c0a4e8d80b65adb069df945/users/47d211e5501f4685bc01e29847e6d498/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:42.564993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.565286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.565569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.593396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] MFA Rules not processed for user `47d211e5501f4685bc01e29847e6d498`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:42.594721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:42.668546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1231/2464] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:42.680046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.682001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:42.746916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01044-ee00-432f-9631-215eeaa59953 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.751111 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12288d-11b4-416c-8219-5fb4a96f9ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.752039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12288d-11b4-416c-8219-5fb4a96f9ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.752318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12288d-11b4-416c-8219-5fb4a96f9ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.752590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12288d-11b4-416c-8219-5fb4a96f9ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.763307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12288d-11b4-416c-8219-5fb4a96f9ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.764751 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12288d-11b4-416c-8219-5fb4a96f9ee2 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.775142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1234/2465] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:42.848267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:42.919746 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74d27a-cff8-4375-aba9-82ef83606119 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:42.924408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:42.925287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:42.925567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:42.925832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] PATH_INFO: `/v3/users/e8b359217aa14103b5d6aea07a525d05` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:42.927105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b359217aa14103b5d6aea07a525d05)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:42.929227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:42.934017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:42.992603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b359217aa14103b5d6aea07a525d05'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:42.993393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b359217aa14103b5d6aea07a525d05'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.000205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b359217aa14103b5d6aea07a525d05'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.000880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b359217aa14103b5d6aea07a525d05'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.013094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b359217aa14103b5d6aea07a525d05'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.026260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1232/2466] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:42 2022] DELETE /identity/v3/users/e8b359217aa14103b5d6aea07a525d05 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:43.031127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:43.031460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:43.031838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:43.051774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:43.053369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:43.066502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:43.120526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1235/2467] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:43.130341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:43.131920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:43.192973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70566-1fea-49c9-8666-c88c90625136 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:43.196718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:43.197366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:43.197648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:43.197927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:43.207948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:43.209385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:43.221316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1233/2468] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:43.504895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:43.506548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:43.521800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c66400-d8f9-4ea5-b81f-738c33bc3034 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:43.526544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:43.527450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:43.527727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:43.528697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] PATH_INFO: `/v3/projects/f76e9a61af6346f3b0e17f84c3599634` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:43.530291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76e9a61af6346f3b0e17f84c3599634)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:43.532259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:43.582962 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76e9a61af6346f3b0e17f84c3599634', 'request_id': 'req-d9d31dd3-52fb-402b-9c62-6794deb83f6c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.589416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76e9a61af6346f3b0e17f84c3599634', 'request_id': 'req-d9d31dd3-52fb-402b-9c62-6794deb83f6c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.595445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76e9a61af6346f3b0e17f84c3599634', 'request_id': 'req-d9d31dd3-52fb-402b-9c62-6794deb83f6c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.608373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] The token cache is being invalidate because project f76e9a61af6346f3b0e17f84c3599634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:43.608659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.609275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.610816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1236/2469] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:43 2022] DELETE /identity/v3/projects/f76e9a61af6346f3b0e17f84c3599634 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:43.614936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:43.624712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242ff15-8237-4f8f-8f56-4e8a10b9b4b1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:43.693231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:43.694374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:43.694803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:43.695239 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] PATH_INFO: `/v3/users/47d211e5501f4685bc01e29847e6d498` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:43.697131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d211e5501f4685bc01e29847e6d498)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:43.700104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:43.734878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:43.755331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d211e5501f4685bc01e29847e6d498'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.755947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d211e5501f4685bc01e29847e6d498'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.762679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d211e5501f4685bc01e29847e6d498'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.763356 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d211e5501f4685bc01e29847e6d498'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.774848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51fa3ee1-46c1-46fc-8f43-fb74594bdc12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d211e5501f4685bc01e29847e6d498'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:43.789292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1234/2470] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:43 2022] DELETE /identity/v3/users/47d211e5501f4685bc01e29847e6d498 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:44.081929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.083488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:44.100119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31dd3-52fb-402b-9c62-6794deb83f6c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.166457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.166457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.166457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.166457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] PATH_INFO: `/v3/projects/01eed35e3c0a4e8d80b65adb069df945` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.167021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01eed35e3c0a4e8d80b65adb069df945)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.167021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.219932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:44.225344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01eed35e3c0a4e8d80b65adb069df945', 'request_id': 'req-59572ba6-cf81-44cc-a09e-dac6ddf377f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:44.235625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01eed35e3c0a4e8d80b65adb069df945', 'request_id': 'req-59572ba6-cf81-44cc-a09e-dac6ddf377f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:44.243571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01eed35e3c0a4e8d80b65adb069df945', 'request_id': 'req-59572ba6-cf81-44cc-a09e-dac6ddf377f2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:44.261294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] The token cache is being invalidate because project 01eed35e3c0a4e8d80b65adb069df945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:44.261639 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:44.262417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-59572ba6-cf81-44cc-a09e-dac6ddf377f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:44.264418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1237/2471] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:44 2022] DELETE /identity/v3/projects/01eed35e3c0a4e8d80b65adb069df945 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:44.269527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.269686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.269981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.295355 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:44.296987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:44.379306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1235/2472] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:44.392708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.393242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.393689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.424906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:44.426685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:44.446138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.453187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1238/2473] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:44.457872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db2d62-7f7f-4fde-92e8-d5d79894a3cd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.463794 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.464845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.465131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.465593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.466957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:44.466957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.468805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.486798 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1236/2474] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:18:44.489051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.498968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a7ef4-e7c2-42cd-b724-542e2b87c0e0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.504246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.504910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.505187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.505741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.506265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.508553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.535722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1239/2475] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:44.540142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.551265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a4669-4d3a-4dfb-b5be-1b4629480d8b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.617920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.618573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.618889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.619163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.619857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.621802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.642685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1237/2476] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:44.647033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.657335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ef015-40f8-428d-aa2a-a8fb7c05391e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.664037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.664702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.664977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.665243 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.665968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.667632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.678482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1240/2477] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:44.684263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.694527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14165f26-0326-4e55-84b2-1eccbff108e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.699579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1668990-4218-4fe2-988a-c2aeb1c89c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.700229 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1668990-4218-4fe2-988a-c2aeb1c89c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.700505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1668990-4218-4fe2-988a-c2aeb1c89c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.700792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1668990-4218-4fe2-988a-c2aeb1c89c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.701524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1668990-4218-4fe2-988a-c2aeb1c89c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.703161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1668990-4218-4fe2-988a-c2aeb1c89c9c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.713944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1238/2478] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:44.718209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.728651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a3c99-328d-4bbd-9e39-e11ed4d756dd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.733500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.734376 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.734652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.734951 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] PATH_INFO: `/v3/projects/c5e0507d78ca4e039f27c0e6b54dead6/users/e9ba97b5f94b40ceb25821f52346d447/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.750897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e0507d78ca4e039f27c0e6b54dead6, user_id=e9ba97b5f94b40ceb25821f52346d447, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.753618 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.767741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1241/2479] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:44 2022] PUT /identity/v3/projects/c5e0507d78ca4e039f27c0e6b54dead6/users/e9ba97b5f94b40ceb25821f52346d447/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:44.777957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.778248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.778518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.801146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:44.809370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] MFA Rules not processed for user `e9ba97b5f94b40ceb25821f52346d447`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:44.811141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:44.879778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1239/2480] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:44.884541 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:44.949500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6462dc-8e48-46c7-a4b9-364db60689a0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:44.955014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:44.955668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:44.955973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:44.956292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:44.956938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:44.959050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:44.987924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1242/2481] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:44.992256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.003765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c8360-6b6a-4167-8096-79492b209b6c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.063908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.064762 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.065104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.065476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.066537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.069479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.091478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1240/2482] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:44 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:45.095551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.107062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fedba1-0bc4-4639-82ba-079d68b2d346 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.113317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.114173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.114454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.114791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.115733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.117805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.129524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1243/2483] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:45.136722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.147532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c8dac-de01-40a7-8d4c-e38e80899f52 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.152303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4a710-8126-4fe9-8df1-9b7db3033c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.152957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4a710-8126-4fe9-8df1-9b7db3033c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.153235 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4a710-8126-4fe9-8df1-9b7db3033c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.153508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4a710-8126-4fe9-8df1-9b7db3033c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.154231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4a710-8126-4fe9-8df1-9b7db3033c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.155852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4a710-8126-4fe9-8df1-9b7db3033c3f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.166732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1241/2484] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:45.173303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.187643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a3f78-4b67-4a5b-876c-a5d5fe834cbf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.193283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.194220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.194500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.194811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] PATH_INFO: `/v3/projects/a4402015c969444294bbd4b492c85208/users/7c0011eaa9ea4c07be9cd2febe3c9f9b/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.210789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4402015c969444294bbd4b492c85208, user_id=7c0011eaa9ea4c07be9cd2febe3c9f9b, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.213833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.230071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1244/2485] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:45 2022] PUT /identity/v3/projects/a4402015c969444294bbd4b492c85208/users/7c0011eaa9ea4c07be9cd2febe3c9f9b/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:45.241893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.242179 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.242451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.268792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] MFA Rules not processed for user `7c0011eaa9ea4c07be9cd2febe3c9f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:45.270044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:45.336662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1242/2486] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:45.343556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.363653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:45.407664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c0a4f-b233-43af-924a-1d910520a6b0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.412532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.413193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.413462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.413749 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.414441 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.416417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.443368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1245/2487] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:45 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:45.447394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.460020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccd0f5-cde5-4cc8-9f38-cb7974408b84 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.466181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:45.527561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.528166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.528576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.528851 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.529586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.531521 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.555141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1243/2488] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:45 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:45.560810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.571790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effab9ed-a477-4a96-9958-0d0dd1a15e35 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.577561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.578312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.578664 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.579035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.579769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.581385 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.593697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1246/2489] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:45.597826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.608049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f73446-8c74-4d39-ac2b-4b7d7462360e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.612914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.613844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.614120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.614385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] PATH_INFO: `/v3/projects/383d5eb9bdfa4cf094cefef46fe88a7d/users/92d9df06b2564fdea3136aa30df167d6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.630905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=383d5eb9bdfa4cf094cefef46fe88a7d, user_id=92d9df06b2564fdea3136aa30df167d6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.633777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.647995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1244/2490] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:45 2022] PUT /identity/v3/projects/383d5eb9bdfa4cf094cefef46fe88a7d/users/92d9df06b2564fdea3136aa30df167d6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:45.651970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.662383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e1067-1f5d-45ba-aecf-3d4d49324b5d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.723622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc7526-88e6-4876-b5ab-b4afcbaf23f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.724925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc7526-88e6-4876-b5ab-b4afcbaf23f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.724925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc7526-88e6-4876-b5ab-b4afcbaf23f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.725200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc7526-88e6-4876-b5ab-b4afcbaf23f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.726065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc7526-88e6-4876-b5ab-b4afcbaf23f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.727701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc7526-88e6-4876-b5ab-b4afcbaf23f8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.738501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1247/2491] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:45.744916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.760134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6c313-3321-4661-9e48-885e91d830fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.765707 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.766585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.766887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.767159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] PATH_INFO: `/v3/projects/383d5eb9bdfa4cf094cefef46fe88a7d/users/92d9df06b2564fdea3136aa30df167d6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.769344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=383d5eb9bdfa4cf094cefef46fe88a7d, user_id=92d9df06b2564fdea3136aa30df167d6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:45.772048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:45.783133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1245/2492] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:45 2022] PUT /identity/v3/projects/383d5eb9bdfa4cf094cefef46fe88a7d/users/92d9df06b2564fdea3136aa30df167d6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:45.793740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.794125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.794475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:45.823401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] MFA Rules not processed for user `92d9df06b2564fdea3136aa30df167d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:45.824634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:45.897748 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1248/2493] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:45.908604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:45.910597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:45.985440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22ee55-921d-42df-b652-1635d00a94ea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:45.991685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:45.992331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:45.992609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:45.992877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:46.003082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:46.004520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:46.014458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1246/2494] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:46.590681 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:46.603421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a489450-6ed2-4259-a6f1-61a7568aad9f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:46.607344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea9053-36bb-4d51-b284-71e6eed070c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:46.607986 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9053-36bb-4d51-b284-71e6eed070c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:46.608265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9053-36bb-4d51-b284-71e6eed070c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:46.608531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9053-36bb-4d51-b284-71e6eed070c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:46.620204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:46.671853 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea9053-36bb-4d51-b284-71e6eed070c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:46.673526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea9053-36bb-4d51-b284-71e6eed070c1 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:46.683817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1249/2495] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:51.368441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:51.370090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:51.475838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c4cf6-9859-4e82-9524-c503f4da7d27 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:51.480422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:51.481302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:51.481576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:51.481845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] PATH_INFO: `/v3/users/a63be4691fe1438991c191520b9af61f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:51.483100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63be4691fe1438991c191520b9af61f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:51.485053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:51.490043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:51.554232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63be4691fe1438991c191520b9af61f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:51.555299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63be4691fe1438991c191520b9af61f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:51.565858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63be4691fe1438991c191520b9af61f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:51.566964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63be4691fe1438991c191520b9af61f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:51.585830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63be4691fe1438991c191520b9af61f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:51.604803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1247/2496] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:51 2022] DELETE /identity/v3/users/a63be4691fe1438991c191520b9af61f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:51.610161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:51.610723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:51.610723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:51.633740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:51.635162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:51.709318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1250/2497] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:51.729170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:51.732081 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:51.819394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab430ca6-04e8-4ebe-93af-4e8a35d59227 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:51.823208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:51.823879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:51.824162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:51.824428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:51.833848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:51.835467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:51.845214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1248/2498] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:52.140844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:52.142296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:52.156200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17b63e-d652-4687-ad4d-1ab87e0feb47 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:52.160916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:52.162941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:52.163228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:52.163557 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] PATH_INFO: `/v3/projects/d19013d24b834a0ba1565d952264b80e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:52.164722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d19013d24b834a0ba1565d952264b80e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:52.166669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:52.221839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd19013d24b834a0ba1565d952264b80e', 'request_id': 'req-4f68009e-8cf0-42a7-b556-e86b7e15101e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.229998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd19013d24b834a0ba1565d952264b80e', 'request_id': 'req-4f68009e-8cf0-42a7-b556-e86b7e15101e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.236549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd19013d24b834a0ba1565d952264b80e', 'request_id': 'req-4f68009e-8cf0-42a7-b556-e86b7e15101e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.252131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] The token cache is being invalidate because project d19013d24b834a0ba1565d952264b80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:52.252411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.253042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.254604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1251/2499] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:52 2022] DELETE /identity/v3/projects/d19013d24b834a0ba1565d952264b80e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:52.259166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:52.269641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b3516-b827-4789-9979-874f52dd8e8a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:52.351387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:52.352618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:52.353027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:52.353387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] PATH_INFO: `/v3/users/5a4368bcd662415fb0bdd50b3d15d42a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:52.355159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a4368bcd662415fb0bdd50b3d15d42a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:52.358586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:52.396261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:52.416509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a4368bcd662415fb0bdd50b3d15d42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.417198 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a4368bcd662415fb0bdd50b3d15d42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.424614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a4368bcd662415fb0bdd50b3d15d42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.425435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a4368bcd662415fb0bdd50b3d15d42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.439086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a4368bcd662415fb0bdd50b3d15d42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.457171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1249/2500] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:52 2022] DELETE /identity/v3/users/5a4368bcd662415fb0bdd50b3d15d42a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:52.676789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:52.678830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:52.693867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68009e-8cf0-42a7-b556-e86b7e15101e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:52.752833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:52.780363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:52.780820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:52.780820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:52.781076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] PATH_INFO: `/v3/users/1cac05c9c0c24c249f65240da0c5434f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:52.782904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cac05c9c0c24c249f65240da0c5434f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:52.784670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:52.821769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:52.823826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:52.845135 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944af0-8eaf-4aae-ad72-fa5d431a90c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:52.850623 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:52.852269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:52.852269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:52.852698 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] PATH_INFO: `/v3/projects/ad663bf380e24e7db211a35742c4c0c7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:52.854084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad663bf380e24e7db211a35742c4c0c7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:52.856169 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:52.890125 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cac05c9c0c24c249f65240da0c5434f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.891167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cac05c9c0c24c249f65240da0c5434f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.892121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cac05c9c0c24c249f65240da0c5434f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.914294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad663bf380e24e7db211a35742c4c0c7', 'request_id': 'req-07ef8504-d515-495d-8203-9a5becc054e3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.919425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cac05c9c0c24c249f65240da0c5434f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.921587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad663bf380e24e7db211a35742c4c0c7', 'request_id': 'req-07ef8504-d515-495d-8203-9a5becc054e3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.929454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad663bf380e24e7db211a35742c4c0c7', 'request_id': 'req-07ef8504-d515-495d-8203-9a5becc054e3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.929873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-55779622-8914-42e1-984e-54d2fdaef957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cac05c9c0c24c249f65240da0c5434f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.945021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] The token cache is being invalidate because project ad663bf380e24e7db211a35742c4c0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:52.945329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.946052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-07ef8504-d515-495d-8203-9a5becc054e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:52.953875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1250/2501] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:52 2022] DELETE /identity/v3/projects/ad663bf380e24e7db211a35742c4c0c7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:52.954071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1252/2502] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:52 2022] DELETE /identity/v3/users/1cac05c9c0c24c249f65240da0c5434f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:52.954178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:52.954178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:52.954178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:52.956514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:52.956821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:52.957163 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:52.979595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:52.979909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:52.980950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:52.981162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.047654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1253/2503] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:53.048425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.049812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.056134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1251/2504] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:53.064010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.065866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.109213 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:53.112804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d84e04-176a-49e2-b6df-9c34fab08816 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.116673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.117331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.117606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.117950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.120145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.131377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:53.146344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920e92c-8fcd-4fc6-98a3-d0cff43b5496 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.150574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aa8c8-56ea-4dae-9a6c-71092690842e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.151321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa8c8-56ea-4dae-9a6c-71092690842e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.151605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa8c8-56ea-4dae-9a6c-71092690842e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.151926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa8c8-56ea-4dae-9a6c-71092690842e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.162459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aa8c8-56ea-4dae-9a6c-71092690842e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.164040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aa8c8-56ea-4dae-9a6c-71092690842e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.175799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1252/2505] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:53.177682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.178293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.178705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.192205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.193835 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.203024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:53.204587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.204701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1254/2506] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:53.215306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1253/2507] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:53.220014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.234896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3535b-c183-4a56-b66d-000991f9af0e service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.241196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.242259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.242377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.242695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.243652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:53.244325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.246658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.267795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1255/2508] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:18:53.269879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.281910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91ec2e-35ef-4ba8-b87f-a6f917ef3b4b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.288342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.289505 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.289910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.290237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.291171 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.293093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.323987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1254/2509] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:53 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:53.328668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.340740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30be32-ea68-4587-a6d2-ced67978f7cf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.425276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.426204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.426586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.427010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.428047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.431369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.454860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1256/2510] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:53 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:53.459671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.476191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f129ec-a9a2-49e4-bcc5-8200ab750419 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.484044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.484831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.485144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.485609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.487016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.488488 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.503308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1255/2511] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:53.508200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.510134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.521773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d1178-a498-4fe7-86c7-21342117fb6b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.524635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.527114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.528493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.528883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.529302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.530090 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.531915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.543469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1257/2512] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:53.547134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accd90e-968d-4fb1-8302-3dafeb2f9d0b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.547887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.554120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.555101 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.555424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.555837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] PATH_INFO: `/v3/projects/7be1a142fcc74c0091165367b122cae0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.557343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be1a142fcc74c0091165367b122cae0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.560579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.561542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd06aa-171e-4307-a8b7-8d7606983663 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.567043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46ed55-3b68-4c84-bc0b-3d70af236d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.567943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ed55-3b68-4c84-bc0b-3d70af236d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.568244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ed55-3b68-4c84-bc0b-3d70af236d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.568523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ed55-3b68-4c84-bc0b-3d70af236d39 admin admin] PATH_INFO: `/v3/projects/a94a18aaeace4a68ac9f55aea94fc9ff/users/9939f47e8db54112b0c8ab91d3785509/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.586077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46ed55-3b68-4c84-bc0b-3d70af236d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94a18aaeace4a68ac9f55aea94fc9ff, user_id=9939f47e8db54112b0c8ab91d3785509, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.589252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46ed55-3b68-4c84-bc0b-3d70af236d39 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.605453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1258/2513] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:53 2022] PUT /identity/v3/projects/a94a18aaeace4a68ac9f55aea94fc9ff/users/9939f47e8db54112b0c8ab91d3785509/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:53.615829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.616115 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.616395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.622014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be1a142fcc74c0091165367b122cae0', 'request_id': 'req-51405f5d-de3b-43c1-9620-f552e38aaa42'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.629282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be1a142fcc74c0091165367b122cae0', 'request_id': 'req-51405f5d-de3b-43c1-9620-f552e38aaa42'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.636082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be1a142fcc74c0091165367b122cae0', 'request_id': 'req-51405f5d-de3b-43c1-9620-f552e38aaa42'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.650305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] MFA Rules not processed for user `9939f47e8db54112b0c8ab91d3785509`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:53.651197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] The token cache is being invalidate because project 7be1a142fcc74c0091165367b122cae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:53.651496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.651682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.652165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.657017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1256/2514] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:53 2022] DELETE /identity/v3/projects/7be1a142fcc74c0091165367b122cae0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:53.658467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.659455 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.713606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:53.748075 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51405f5d-de3b-43c1-9620-f552e38aaa42 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.754589 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1259/2515] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:53.756023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.757077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.757358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.757627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] PATH_INFO: `/v3/users/b9aa1b465efa4dd7b76a4d2a64c3f7ff` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.759291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9aa1b465efa4dd7b76a4d2a64c3f7ff)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.761288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.766261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.767854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.781680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:53.825145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9aa1b465efa4dd7b76a4d2a64c3f7ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.826109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9aa1b465efa4dd7b76a4d2a64c3f7ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.836533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9aa1b465efa4dd7b76a4d2a64c3f7ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.837284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9aa1b465efa4dd7b76a4d2a64c3f7ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.855388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9aa1b465efa4dd7b76a4d2a64c3f7ff'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:53.871415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bde791-0878-42c0-be8e-da31b3058b2a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:53.875125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1257/2516] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:53 2022] DELETE /identity/v3/users/b9aa1b465efa4dd7b76a4d2a64c3f7ff => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:53.878292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:53.878292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:53.878292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:53.878292 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:53.880356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:53.896847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:53.898979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937753f-4994-49c9-aedd-35f708e718bc service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:53.910492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1260/2517] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:53.932845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:53.934803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.034339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338952b-cc6f-419f-bcc3-80ed6a50f690 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.042017 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.042976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.043254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.043740 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] PATH_INFO: `/v3/users/9939f47e8db54112b0c8ab91d3785509` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.044953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9939f47e8db54112b0c8ab91d3785509)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.046811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.104982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9939f47e8db54112b0c8ab91d3785509'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.105652 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9939f47e8db54112b0c8ab91d3785509'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.113507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9939f47e8db54112b0c8ab91d3785509'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.114178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9939f47e8db54112b0c8ab91d3785509'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.127587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9939f47e8db54112b0c8ab91d3785509'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.142989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1258/2518] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:53 2022] DELETE /identity/v3/users/9939f47e8db54112b0c8ab91d3785509 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:54.148643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.148934 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.149208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.168864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:54.170465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.220699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.222958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.243485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:54.255236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1261/2519] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:54.269167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.270718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.314105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0df515-b076-43dd-a809-7933738ac7f2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.319332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.320243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.320522 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.320788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] PATH_INFO: `/v3/projects/06b3f5216b00470e8dd8433c59db39e0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.322141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b3f5216b00470e8dd8433c59db39e0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.324564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.334615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbc015-07f7-4dea-92c8-f83e66d2cb96 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.338870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3389d4-8b7a-420d-9623-022b1ea7755f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.339636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3389d4-8b7a-420d-9623-022b1ea7755f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.339989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3389d4-8b7a-420d-9623-022b1ea7755f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.340370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3389d4-8b7a-420d-9623-022b1ea7755f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.351966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3389d4-8b7a-420d-9623-022b1ea7755f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.353607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3389d4-8b7a-420d-9623-022b1ea7755f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.364394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1262/2520] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:54.387112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b3f5216b00470e8dd8433c59db39e0', 'request_id': 'req-9b83b80e-4723-4d20-b41d-62b581dbbe9c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.394439 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b3f5216b00470e8dd8433c59db39e0', 'request_id': 'req-9b83b80e-4723-4d20-b41d-62b581dbbe9c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.401340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b3f5216b00470e8dd8433c59db39e0', 'request_id': 'req-9b83b80e-4723-4d20-b41d-62b581dbbe9c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.414104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] The token cache is being invalidate because project 06b3f5216b00470e8dd8433c59db39e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:54.414563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.415381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9b83b80e-4723-4d20-b41d-62b581dbbe9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.417431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1259/2521] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:54 2022] DELETE /identity/v3/projects/06b3f5216b00470e8dd8433c59db39e0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:54.423146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.423430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.423708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.445780 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:54.446804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.473055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:54.524382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1263/2522] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:54.532678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.533123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.533123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.554382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:54.555641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.565194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1260/2523] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:54.570160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.580466 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4763b-ac2f-47b0-b68f-17055b56e18b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.584914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.585584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.585862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.586132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.586901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:54.587209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.589465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.603964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1264/2524] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:18:54.611331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.631830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3214c-b6e0-423a-b000-fbea6ffdd4a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.639283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.640092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.640329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.640599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.641437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.643744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.680788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1261/2525] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:54.681436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.683372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.685273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.701230 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d202467-6b62-47fc-9ffd-a55ff4413a50 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.765701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d0f0b-5bbd-499c-8621-b765a5603ab3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.769220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.770108 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.770387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.770663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] PATH_INFO: `/v3/projects/a94a18aaeace4a68ac9f55aea94fc9ff` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.771928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94a18aaeace4a68ac9f55aea94fc9ff)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.772393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.773088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.773367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.773637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.773831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.774336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.776581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.802896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1262/2526] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:54.806212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.813426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:54.818668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567fde-f426-4b87-af0f-ca0de4dd11c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.829507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.830921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.831341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.831824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.833313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.839959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.841071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94a18aaeace4a68ac9f55aea94fc9ff', 'request_id': 'req-2247efa4-080c-4204-8060-c570dfa07bed'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.849409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94a18aaeace4a68ac9f55aea94fc9ff', 'request_id': 'req-2247efa4-080c-4204-8060-c570dfa07bed'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.851700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1263/2527] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:54.855761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.856896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94a18aaeace4a68ac9f55aea94fc9ff', 'request_id': 'req-2247efa4-080c-4204-8060-c570dfa07bed'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.865843 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcaadbd-23ff-41ab-af52-903ec0cee24d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.873325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] The token cache is being invalidate because project a94a18aaeace4a68ac9f55aea94fc9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:54.873655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.874394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2247efa4-080c-4204-8060-c570dfa07bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:54.876611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1265/2528] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:54 2022] DELETE /identity/v3/projects/a94a18aaeace4a68ac9f55aea94fc9ff => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:54.881985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0c79-adf8-4e9e-a91f-23205448777a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.882297 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0c79-adf8-4e9e-a91f-23205448777a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.882620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0c79-adf8-4e9e-a91f-23205448777a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.903589 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-416c0c79-adf8-4e9e-a91f-23205448777a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:54.904947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-416c0c79-adf8-4e9e-a91f-23205448777a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.928408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.929250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.929528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.929838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.930785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.932594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:54.943571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1264/2529] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:54.948161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:54.950038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:54.956063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:54.965567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb90ab-d231-4736-91d0-8e771a1a274e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:54.971785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58322aec-d746-426f-8630-618fc06cdbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:54.972896 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322aec-d746-426f-8630-618fc06cdbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.973258 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322aec-d746-426f-8630-618fc06cdbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.973542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322aec-d746-426f-8630-618fc06cdbbf admin admin] PATH_INFO: `/v3/projects/93fb44c232ec4f47bfc1f6bae046554d/users/fd56c245b0134f67bc5ca5d2b7e29a5e/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.986501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1266/2530] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:54.994596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58322aec-d746-426f-8630-618fc06cdbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=93fb44c232ec4f47bfc1f6bae046554d, user_id=fd56c245b0134f67bc5ca5d2b7e29a5e, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:54.996624 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:54.996941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:54.997253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:54.998106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58322aec-d746-426f-8630-618fc06cdbbf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.029763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1265/2531] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:54 2022] PUT /identity/v3/projects/93fb44c232ec4f47bfc1f6bae046554d/users/fd56c245b0134f67bc5ca5d2b7e29a5e/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:55.030137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:55.030137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:55.031939 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.032273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.032573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.063132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] MFA Rules not processed for user `fd56c245b0134f67bc5ca5d2b7e29a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:55.064893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:55.106928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1267/2532] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:55.109209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.121533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c5a60-fe97-471e-a9cd-2a7a519e3a5a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.126891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.127587 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.127870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.128150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.129501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:55.129501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.131076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.146011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1268/2533] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:18:55.148216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1266/2534] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:55.150487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.153496 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.165452 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb73b6-f7be-4ea2-b576-4736462aa87f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.171084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.171206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b314642-34b6-4c9e-a552-69958a6f0eac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.171798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.172114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.172392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.173159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.175295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.176037 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.176730 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.176999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.177271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.178044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.180021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.216148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1267/2535] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:55.216370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1269/2536] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:55.216892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.217712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.265350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:55.301848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597171d7-1fa0-4958-a024-099001a90da8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.302060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06196d14-8d8e-4b83-8df3-764544fa0d89 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.307805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.308244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.308662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.309003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.309122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.309291 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.309399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.309683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.310190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.310497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.312625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.313157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.338384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1270/2537] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:55.343364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1268/2538] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:55.343727 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.347827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.358407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c388e27-b7cc-4d23-a053-23a557d2a2ab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.360490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9794349-a353-4cff-900a-fc985ef279c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.364489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.365370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.365686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.365978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.366112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.366661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.366873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.367001 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.367265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.368034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.368882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.369677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.383022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1271/2539] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:55.387076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1269/2540] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:55.388013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.389958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.401879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c92eb-f960-431a-9839-5120d9acc883 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.402609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269d755-126c-4466-9e94-f2795839ee0d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.408747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.410280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.410455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.410635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.411053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] PATH_INFO: `/v3/projects/9ab346166b8d4a4c935454f88fbb0e97/users/af4ae72b52044d7791254bcc33e414cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.411161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.411394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.411674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.412644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.414503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.427901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1270/2541] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:55.432574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.433872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab346166b8d4a4c935454f88fbb0e97, user_id=af4ae72b52044d7791254bcc33e414cd, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.437261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.445270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f644d-36ee-4777-9356-72f985b701b1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.452696 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e587af1-37fc-4ecc-9dcb-94b4eb38b0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.454150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587af1-37fc-4ecc-9dcb-94b4eb38b0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.454799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1272/2542] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:55 2022] PUT /identity/v3/projects/9ab346166b8d4a4c935454f88fbb0e97/users/af4ae72b52044d7791254bcc33e414cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:55.455274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587af1-37fc-4ecc-9dcb-94b4eb38b0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.455558 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e587af1-37fc-4ecc-9dcb-94b4eb38b0cb admin admin] PATH_INFO: `/v3/projects/74aae67b92a142d68bc7302a97f7f84b/users/12888093e02c4d6f91326e261218698c/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.459337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.466625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:55.476028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e587af1-37fc-4ecc-9dcb-94b4eb38b0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=74aae67b92a142d68bc7302a97f7f84b, user_id=12888093e02c4d6f91326e261218698c, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.479090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e587af1-37fc-4ecc-9dcb-94b4eb38b0cb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.496372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1271/2543] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:55 2022] PUT /identity/v3/projects/74aae67b92a142d68bc7302a97f7f84b/users/12888093e02c4d6f91326e261218698c/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:55.508010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.508340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.508662 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.542989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd31be2-8211-4977-94f9-aca0e272e731 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.543339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] MFA Rules not processed for user `12888093e02c4d6f91326e261218698c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:55.544995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:55.617025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.617813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.618186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.618553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.619540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.621467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.636902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1273/2544] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:55.638481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.639703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1272/2545] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:55.653333 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.655010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:55.657974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199125c-fb35-49b9-83fb-4f07e298699c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.664576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.665649 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.665950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.666262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] PATH_INFO: `/v3/projects/9ab346166b8d4a4c935454f88fbb0e97/users/af4ae72b52044d7791254bcc33e414cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.669060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab346166b8d4a4c935454f88fbb0e97, user_id=af4ae72b52044d7791254bcc33e414cd, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.672194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.687867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1274/2546] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:55 2022] PUT /identity/v3/projects/9ab346166b8d4a4c935454f88fbb0e97/users/af4ae72b52044d7791254bcc33e414cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:55.691530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:55.693774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:55.755957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:55.762346 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e27a2-e90e-4714-a1af-54d7594e9e88 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.777795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe29ff-1a8f-4376-bf2e-a92e65c62c7c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:55.782985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.783904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.784183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.784453 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] PATH_INFO: `/v3/users/e9ba97b5f94b40ceb25821f52346d447` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.785711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ba97b5f94b40ceb25821f52346d447)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.788073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.822043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4d404-3c63-4032-888d-1027313c7163 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:55.822937 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4d404-3c63-4032-888d-1027313c7163 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.823273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4d404-3c63-4032-888d-1027313c7163 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.823594 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4d404-3c63-4032-888d-1027313c7163 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.833460 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:55.844579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ba97b5f94b40ceb25821f52346d447'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:55.845353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ba97b5f94b40ceb25821f52346d447'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:55.846363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ba97b5f94b40ceb25821f52346d447'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:55.859023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ba97b5f94b40ceb25821f52346d447'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:55.866235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ddd31110-13ce-4799-94ba-c1f66e6fc9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ba97b5f94b40ceb25821f52346d447'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:55.884106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1275/2547] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:55 2022] DELETE /identity/v3/users/e9ba97b5f94b40ceb25821f52346d447 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:55.884106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.884106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.884106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.895145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4d404-3c63-4032-888d-1027313c7163 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:55.896765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4d404-3c63-4032-888d-1027313c7163 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:55.906349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1273/2548] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:55.907629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:55.907920 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:55.908182 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:55.916998 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] MFA Rules not processed for user `af4ae72b52044d7791254bcc33e414cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:55.918691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:55.945449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:55.947292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:56.021445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1276/2549] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:56.021924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1274/2550] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:56.033028 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:56.033687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:56.035008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:56.035165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:56.103989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40f2ba-5f5f-4b4b-8f57-b79d2a9088de None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:56.104380 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ceb87d-28df-4802-9ce3-27d0bdb80683 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:56.108687 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:56.109111 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:56.109718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:56.109890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:56.110177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:56.110269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:56.110617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:56.110713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:56.121603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:56.123263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:56.123452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:56.125177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:56.130278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:56.139023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1277/2551] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:56.147387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1275/2552] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:18:56.518179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:56.520753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:56.552098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44b55a-13c3-4e21-8626-91c1c1e2a594 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:56.559200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:56.560553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:56.560935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:56.561338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] PATH_INFO: `/v3/projects/c5e0507d78ca4e039f27c0e6b54dead6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:56.564178 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e0507d78ca4e039f27c0e6b54dead6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:56.566490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:56.618966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:56.633801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e0507d78ca4e039f27c0e6b54dead6', 'request_id': 'req-1b165b77-f533-4d31-9a61-f1f131cd979b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.641490 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e0507d78ca4e039f27c0e6b54dead6', 'request_id': 'req-1b165b77-f533-4d31-9a61-f1f131cd979b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.654828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e0507d78ca4e039f27c0e6b54dead6', 'request_id': 'req-1b165b77-f533-4d31-9a61-f1f131cd979b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.667386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] The token cache is being invalidate because project c5e0507d78ca4e039f27c0e6b54dead6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:56.667675 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.668383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.670337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1278/2553] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:56 2022] DELETE /identity/v3/projects/c5e0507d78ca4e039f27c0e6b54dead6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:56.675585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:56.687693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8032d-8b12-463b-9a0d-b995200c7a2a service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:56.715464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:56.717761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:56.767098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:56.768617 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:56.769059 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:56.769963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] PATH_INFO: `/v3/users/7c0011eaa9ea4c07be9cd2febe3c9f9b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:56.771743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0011eaa9ea4c07be9cd2febe3c9f9b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:56.774421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:56.809160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b165b77-f533-4d31-9a61-f1f131cd979b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:56.814161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:56.816428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:56.816428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:56.816428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] PATH_INFO: `/v3/users/fd56c245b0134f67bc5ca5d2b7e29a5e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:56.817826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd56c245b0134f67bc5ca5d2b7e29a5e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:56.819891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:56.840639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0011eaa9ea4c07be9cd2febe3c9f9b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.841468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0011eaa9ea4c07be9cd2febe3c9f9b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.851295 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0011eaa9ea4c07be9cd2febe3c9f9b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.852250 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0011eaa9ea4c07be9cd2febe3c9f9b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.868414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-51188135-b09d-463b-b574-5372dbebeb21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0011eaa9ea4c07be9cd2febe3c9f9b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.874542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd56c245b0134f67bc5ca5d2b7e29a5e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.876364 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd56c245b0134f67bc5ca5d2b7e29a5e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.877338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd56c245b0134f67bc5ca5d2b7e29a5e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.890700 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1276/2554] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:56 2022] DELETE /identity/v3/users/7c0011eaa9ea4c07be9cd2febe3c9f9b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:56.903030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd56c245b0134f67bc5ca5d2b7e29a5e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.911869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd56c245b0134f67bc5ca5d2b7e29a5e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:56.936117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1279/2555] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:56 2022] DELETE /identity/v3/users/fd56c245b0134f67bc5ca5d2b7e29a5e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:56.938998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:56.939674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:56.940309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:56.965655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:56.967149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:57.047279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1277/2556] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:57.058512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:57.060908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:57.115463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:57.134566 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678fb6-9d05-400f-b353-02423c91b381 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:57.140645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:57.140645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:57.140880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:57.141034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:57.156024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:57.157467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:57.170346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1280/2557] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:57.320465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:57.322194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:57.338085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe889be-da6a-4eeb-8012-33cba1bbd8e6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:57.343799 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:57.344985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:57.345683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:57.345992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] PATH_INFO: `/v3/projects/a4402015c969444294bbd4b492c85208` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:57.350938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4402015c969444294bbd4b492c85208)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:57.354347 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:57.385743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:57.429952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4402015c969444294bbd4b492c85208', 'request_id': 'req-fd76215c-4ca7-46f8-8562-00f1645241f1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.436952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4402015c969444294bbd4b492c85208', 'request_id': 'req-fd76215c-4ca7-46f8-8562-00f1645241f1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.443500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4402015c969444294bbd4b492c85208', 'request_id': 'req-fd76215c-4ca7-46f8-8562-00f1645241f1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.461323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] The token cache is being invalidate because project a4402015c969444294bbd4b492c85208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:57.461708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.462241 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.464261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1278/2558] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:57 2022] DELETE /identity/v3/projects/a4402015c969444294bbd4b492c85208 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:57.468518 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:57.480248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce110-739a-4b01-a23f-93311ed01397 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:57.565562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:57.566099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:57.566767 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:57.567238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] PATH_INFO: `/v3/users/92d9df06b2564fdea3136aa30df167d6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:57.567717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:57.569598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:57.573210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d9df06b2564fdea3136aa30df167d6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:57.576185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:57.596418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76215c-4ca7-46f8-8562-00f1645241f1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:57.610958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:57.612574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:57.613113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:57.613611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] PATH_INFO: `/v3/projects/93fb44c232ec4f47bfc1f6bae046554d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:57.616113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93fb44c232ec4f47bfc1f6bae046554d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:57.619243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:57.667315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d9df06b2564fdea3136aa30df167d6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.668482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d9df06b2564fdea3136aa30df167d6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.671141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d9df06b2564fdea3136aa30df167d6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.686969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d9df06b2564fdea3136aa30df167d6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.695621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d9df06b2564fdea3136aa30df167d6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.702157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93fb44c232ec4f47bfc1f6bae046554d', 'request_id': 'req-9c089368-4037-4f7c-a302-72dc3f4c6140'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.715599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93fb44c232ec4f47bfc1f6bae046554d', 'request_id': 'req-9c089368-4037-4f7c-a302-72dc3f4c6140'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.720573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1281/2559] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:57 2022] DELETE /identity/v3/users/92d9df06b2564fdea3136aa30df167d6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:57.732809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93fb44c232ec4f47bfc1f6bae046554d', 'request_id': 'req-9c089368-4037-4f7c-a302-72dc3f4c6140'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.756396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] The token cache is being invalidate because project 93fb44c232ec4f47bfc1f6bae046554d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:57.756957 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.757704 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.759618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1279/2560] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:57 2022] DELETE /identity/v3/projects/93fb44c232ec4f47bfc1f6bae046554d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:57.765148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:57.766838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:57.866034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9da28f-dcc7-41a8-afcb-f6a1db6a8b21 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:57.874953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:57.876253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:57.876760 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:57.877177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] PATH_INFO: `/v3/users/af4ae72b52044d7791254bcc33e414cd` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:57.878830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af4ae72b52044d7791254bcc33e414cd)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:57.882699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:57.890864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:57.963648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af4ae72b52044d7791254bcc33e414cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.964464 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af4ae72b52044d7791254bcc33e414cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.965086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af4ae72b52044d7791254bcc33e414cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.980356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af4ae72b52044d7791254bcc33e414cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:57.989360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af4ae72b52044d7791254bcc33e414cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.008465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1282/2561] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:57 2022] DELETE /identity/v3/users/af4ae72b52044d7791254bcc33e414cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:58.098000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.100165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:58.206609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c089368-4037-4f7c-a302-72dc3f4c6140 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.254044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:58.296961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.298170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.298570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.299050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] PATH_INFO: `/v3/projects/383d5eb9bdfa4cf094cefef46fe88a7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.303935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] RBAC: Authorizing `identity:delete_project(project_id=383d5eb9bdfa4cf094cefef46fe88a7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.307378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.379791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383d5eb9bdfa4cf094cefef46fe88a7d', 'request_id': 'req-e009d71b-23f1-42ba-a307-26c03311dbea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.386738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383d5eb9bdfa4cf094cefef46fe88a7d', 'request_id': 'req-e009d71b-23f1-42ba-a307-26c03311dbea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.394300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383d5eb9bdfa4cf094cefef46fe88a7d', 'request_id': 'req-e009d71b-23f1-42ba-a307-26c03311dbea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.409381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] The token cache is being invalidate because project 383d5eb9bdfa4cf094cefef46fe88a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:58.410080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.410999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e009d71b-23f1-42ba-a307-26c03311dbea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.411128 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.412694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:58.413362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1280/2562] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:58 2022] DELETE /identity/v3/projects/383d5eb9bdfa4cf094cefef46fe88a7d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:58.423933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de19c2-7a3a-4cc4-823a-925f7fab95e5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.424219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de19c2-7a3a-4cc4-823a-925f7fab95e5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.424502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de19c2-7a3a-4cc4-823a-925f7fab95e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.429602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0c175-794d-4d29-83cc-e74a46deda23 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.446025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d9de19c2-7a3a-4cc4-823a-925f7fab95e5 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:58.447329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d9de19c2-7a3a-4cc4-823a-925f7fab95e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:58.499456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.500422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.500725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.500997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] PATH_INFO: `/v3/projects/9ab346166b8d4a4c935454f88fbb0e97` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.502290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ab346166b8d4a4c935454f88fbb0e97)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.504331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.546078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1281/2563] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:58.556683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.557033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.557278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.558562 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ab346166b8d4a4c935454f88fbb0e97', 'request_id': 'req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.568371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ab346166b8d4a4c935454f88fbb0e97', 'request_id': 'req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.576635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:58.578055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:58.583058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ab346166b8d4a4c935454f88fbb0e97', 'request_id': 'req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.599216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] The token cache is being invalidate because project 9ab346166b8d4a4c935454f88fbb0e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:18:58.599710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.601036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a6c3a5e-5a42-4705-8f99-6ca734d9257e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:58.602812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1283/2564] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:18:58 2022] DELETE /identity/v3/projects/9ab346166b8d4a4c935454f88fbb0e97 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:58.608480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308221-ed71-46e9-bb36-0cdb9cbb1d68 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.609040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308221-ed71-46e9-bb36-0cdb9cbb1d68 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.609330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308221-ed71-46e9-bb36-0cdb9cbb1d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.636809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-18308221-ed71-46e9-bb36-0cdb9cbb1d68 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:58.638890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-18308221-ed71-46e9-bb36-0cdb9cbb1d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:58.649868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:58.661854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1282/2565] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:58.667981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.673891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:58.678492 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4000e-ae00-4612-87fc-fdbfc8ccbbb7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.686265 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.686988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.687283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.687786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.688758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:58.689308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.690942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.703479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1283/2566] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:18:58.707643 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.718038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220ba91-63a9-4ebb-9f56-d3c0debb0f0b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.724180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1284/2567] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:58.725015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.726272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.726606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.727001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.727097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.727294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.727570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.727802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.730184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.746809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:58.748266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:58.758734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1285/2568] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:58.762946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.763205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1284/2569] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:58.767586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.773491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae9ab-c238-4705-9ef8-8930a8cd1c80 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.847340 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.848063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.848344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.848620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.849433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:18:58.849752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.851471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.857911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00465e8e-fb77-4a29-9da0-35bd948ab323 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.863966 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.864310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1286/2570] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:18:58.868292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.868292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.868292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.868292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.868292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.871963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.882723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab1d2-5745-4a0c-b0dd-1089de17faa9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.888054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.888756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.889071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.889348 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.890058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.892105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.893088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1285/2571] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:58.897732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.908574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30666d00-b797-4ccd-967b-41ee0159036f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.913600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:58.914382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:58.914905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:58.915411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:58.916332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:58.918658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:58.925786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1287/2572] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/projects => generated 355 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:58.931502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.933435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1286/2573] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:58.937468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:58.948736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b1d0e-2f90-42b9-9134-6e0cf2a6a3d1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:58.998588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349115b9-6650-4230-8b73-05a3f7db074a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.003640 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.004365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.004903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.004903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.005657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.007647 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.016212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.017413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.017717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.018023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.018987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.021339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.031997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1288/2574] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:58 2022] POST /identity/v3/users => generated 278 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:18:59.037039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1287/2575] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:59.037210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.041527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.048657 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517991f2-a38e-4646-aeb8-3e097cb27fd3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.053864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.054629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.054943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.055272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.055420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c6a7d-8bdd-46e7-84d9-a376954e77bd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.056132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.057866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.060309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb812a-0607-4485-8779-e66f3c314a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.061293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb812a-0607-4485-8779-e66f3c314a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.061607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb812a-0607-4485-8779-e66f3c314a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.061884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb812a-0607-4485-8779-e66f3c314a4e admin admin] PATH_INFO: `/v3/projects/0000a4cc30c0473c8dc93777614a4b3c/users/1f4b74dca9ca43928ba92024b6b3f5f6/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.063914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:59.071542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1289/2576] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:18:59.078312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.079626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb812a-0607-4485-8779-e66f3c314a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0000a4cc30c0473c8dc93777614a4b3c, user_id=1f4b74dca9ca43928ba92024b6b3f5f6, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.082334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb812a-0607-4485-8779-e66f3c314a4e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.090428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea15b46-89c7-4cdf-b455-ddc77099ce7a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.099465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1288/2577] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:59 2022] PUT /identity/v3/projects/0000a4cc30c0473c8dc93777614a4b3c/users/1f4b74dca9ca43928ba92024b6b3f5f6/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:59.112163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.112637 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.113067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.154919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.155797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.156071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.156352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.156995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] MFA Rules not processed for user `1f4b74dca9ca43928ba92024b6b3f5f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:59.157145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.158797 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.159058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:59.173510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1290/2578] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:59.174030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:59.178286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.188976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1546b6-3df8-40b4-bb86-97a9b002327f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.194198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd497544-8366-4a05-9fbd-77592582bbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.195155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd497544-8366-4a05-9fbd-77592582bbc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.195435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd497544-8366-4a05-9fbd-77592582bbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.195712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd497544-8366-4a05-9fbd-77592582bbc3 admin admin] PATH_INFO: `/v3/projects/362682a090514b81a24d2246460f6993/users/e53a2143327b4c418cabf6675f64f8a3/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.211698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd497544-8366-4a05-9fbd-77592582bbc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=362682a090514b81a24d2246460f6993, user_id=e53a2143327b4c418cabf6675f64f8a3, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.214692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd497544-8366-4a05-9fbd-77592582bbc3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.234867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1291/2579] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:59 2022] PUT /identity/v3/projects/362682a090514b81a24d2246460f6993/users/e53a2143327b4c418cabf6675f64f8a3/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:59.243445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.243732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.244009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.256865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1289/2580] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:59.268034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.269827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:59.286981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] MFA Rules not processed for user `e53a2143327b4c418cabf6675f64f8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:59.288823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:59.338709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfbd2e-0044-436e-a748-215969756632 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.379652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1292/2581] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:18:59.386273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.416654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.418317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.419043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.420184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.476239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb6682-dfa7-4532-9799-62afd5d03dc1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.481919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.482769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.483091 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.483408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.484211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.486338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.492282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:59.522472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.523878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.525594 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1293/2582] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:18:59 2022] POST /identity/v3/projects => generated 357 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:59.529915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.535003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1290/2583] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:18:59.556953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.569465 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1d2cd-c404-46b0-a61a-6fa8abd96883 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.591926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:18:59.610361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2cd2b-e8d8-419f-b348-c57761f8d62d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.619214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.620033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.620328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.620606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.621487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.624632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.658874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.660040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.660489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.660797 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] PATH_INFO: `/v3/users/1f4b74dca9ca43928ba92024b6b3f5f6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.662640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4b74dca9ca43928ba92024b6b3f5f6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.667479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1294/2584] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:18:59 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:18:59.667800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.671268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.690061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1c801-014c-4c52-81c2-589d0ffa6307 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.698451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.699656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.700168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.700656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.701831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.705070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.720925 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1295/2585] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:59.729446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.733705 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4b74dca9ca43928ba92024b6b3f5f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:59.734497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4b74dca9ca43928ba92024b6b3f5f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:59.744406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4b74dca9ca43928ba92024b6b3f5f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:59.745461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4b74dca9ca43928ba92024b6b3f5f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:59.749443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15743b-2e2d-4188-b0df-48f20fd0dd62 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.757869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.759548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.760013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.760468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] PATH_INFO: `/v3/projects/3dadd1e58ecc4e629fbd90fc990a8e24/users/4a89470f01bc442eaa01ec7629b2471d/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.763686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d0449fd5-e7f0-451d-b973-9b460ed2ec9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4b74dca9ca43928ba92024b6b3f5f6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:18:59.783082 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1291/2586] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:18:59 2022] DELETE /identity/v3/users/1f4b74dca9ca43928ba92024b6b3f5f6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:18:59.790020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dadd1e58ecc4e629fbd90fc990a8e24, user_id=4a89470f01bc442eaa01ec7629b2471d, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.790353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.790915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.791387 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.792873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.810991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1296/2587] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:59 2022] PUT /identity/v3/projects/3dadd1e58ecc4e629fbd90fc990a8e24/users/4a89470f01bc442eaa01ec7629b2471d/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:59.815956 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.819029 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:59.821914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:18:59.823680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:59.896502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e974d-68f9-4226-b91e-566a9fb6ee09 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.902965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.903691 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.903981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.904266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.905098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.906645 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.912153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1292/2588] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:18:59.921274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1297/2589] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:18:59.924226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.925732 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:18:59.926274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:18:59.941827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1bdd5-e2cc-4c7d-943e-c699afaceae8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:18:59.947820 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47abeaf-b645-4ad0-a5d5-120a6bb4a71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:18:59.948858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47abeaf-b645-4ad0-a5d5-120a6bb4a71f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.949653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47abeaf-b645-4ad0-a5d5-120a6bb4a71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.950170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47abeaf-b645-4ad0-a5d5-120a6bb4a71f admin admin] PATH_INFO: `/v3/projects/3dadd1e58ecc4e629fbd90fc990a8e24/users/4a89470f01bc442eaa01ec7629b2471d/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.952872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47abeaf-b645-4ad0-a5d5-120a6bb4a71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dadd1e58ecc4e629fbd90fc990a8e24, user_id=4a89470f01bc442eaa01ec7629b2471d, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:18:59.955684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47abeaf-b645-4ad0-a5d5-120a6bb4a71f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:18:59.970084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1298/2590] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:18:59 2022] PUT /identity/v3/projects/3dadd1e58ecc4e629fbd90fc990a8e24/users/4a89470f01bc442eaa01ec7629b2471d/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:18:59.981238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:18:59.981548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:18:59.981804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:18:59.982890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:00.002452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9d438-f709-4877-9ff9-2aab4d952dac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.014882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] MFA Rules not processed for user `4a89470f01bc442eaa01ec7629b2471d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:00.016278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:00.031570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:00.072127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.072909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.073051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.073335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.100709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1299/2591] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:00.113435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:00.115434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:00.179591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:00.181088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:00.186513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fdff3-41a6-4d43-9867-87d4ca55c68b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.190281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.191007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.191236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.191491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.191750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1293/2592] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:00.201893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:00.203414 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:00.215360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1300/2593] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:00.296716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:00.306946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4a284-c792-4cde-96f6-100da32b2e19 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.310909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.311581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.311858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.312129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.314096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:00.389783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:00.391573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c976c6d-87c4-42dd-84d0-172dfb196a99 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:00.402449 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1294/2594] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:00.505891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:00.507650 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:00.530778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bc2b5-a4aa-40b0-b9da-062a6d787534 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.540252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.541396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.541752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.542207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] PATH_INFO: `/v3/projects/0000a4cc30c0473c8dc93777614a4b3c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.544024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0000a4cc30c0473c8dc93777614a4b3c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:00.546572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:00.615875 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0000a4cc30c0473c8dc93777614a4b3c', 'request_id': 'req-5f3fd46d-f736-4448-92ea-5829a26b4290'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:00.623003 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0000a4cc30c0473c8dc93777614a4b3c', 'request_id': 'req-5f3fd46d-f736-4448-92ea-5829a26b4290'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:00.629286 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0000a4cc30c0473c8dc93777614a4b3c', 'request_id': 'req-5f3fd46d-f736-4448-92ea-5829a26b4290'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:00.644067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] The token cache is being invalidate because project 0000a4cc30c0473c8dc93777614a4b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:00.644355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:00.645070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f3fd46d-f736-4448-92ea-5829a26b4290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:00.646999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1301/2595] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:00 2022] DELETE /identity/v3/projects/0000a4cc30c0473c8dc93777614a4b3c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:00.653195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.653529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.653861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.674124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:00.675440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:00.732491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:00.758922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1295/2596] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:00.767924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.768331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.769834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.789083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:00.790334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:00.800362 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1302/2597] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:00.807262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:00.821354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04fadc-5df6-4fbb-a6f9-9a13277b3104 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.827854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.828552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.828824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.829092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.829832 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:19:00.830127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:00.831860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:00.847019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1296/2598] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:19:00.852335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:00.862542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbcadb-9ed1-4eed-a056-3312b7b1f2fb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.869710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.870563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.871001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.871377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:00.872307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:00.875007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:00.914488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1303/2599] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:00 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:00.921100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:00.933076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc0afd-0f0f-47e0-a500-16343b1e562f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:00.997991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:00.999251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:00.999615 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:00.999996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:01.001049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:01.003856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:01.030576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1297/2600] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:00 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:19:01.034877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:01.049949 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396d5c-78fe-44f8-8a56-55e3e1603186 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:01.056550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:01.057262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:01.057538 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:01.057809 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:01.058533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:01.060532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:01.062387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:01.078534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1304/2601] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:01.081003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:01.092627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9ae76-9769-4d17-b4fc-f665b9624fca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:01.098699 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893ed05-341f-4921-b258-b6cffcc99376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:01.099575 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893ed05-341f-4921-b258-b6cffcc99376 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:01.100193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893ed05-341f-4921-b258-b6cffcc99376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:01.100586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893ed05-341f-4921-b258-b6cffcc99376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:01.101313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893ed05-341f-4921-b258-b6cffcc99376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:01.102983 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893ed05-341f-4921-b258-b6cffcc99376 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:01.115322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1298/2602] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:01.119312 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:01.130296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a396c74-e470-4beb-bfff-f204671ed285 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:01.135338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:01.136264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:01.136584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:01.136843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] PATH_INFO: `/v3/projects/630246769a674d5b9c5437dfa07b3903/users/3f41efe7f64f41698b5f9ce002c7da71/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:01.154446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630246769a674d5b9c5437dfa07b3903, user_id=3f41efe7f64f41698b5f9ce002c7da71, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:01.157455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:01.172264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1305/2603] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:01 2022] PUT /identity/v3/projects/630246769a674d5b9c5437dfa07b3903/users/3f41efe7f64f41698b5f9ce002c7da71/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:01.178550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fc625-ee1e-4a75-8651-55ebca743ed6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:01.178985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fc625-ee1e-4a75-8651-55ebca743ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:01.179341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703fc625-ee1e-4a75-8651-55ebca743ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:01.202781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-703fc625-ee1e-4a75-8651-55ebca743ed6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:01.204450 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-703fc625-ee1e-4a75-8651-55ebca743ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:01.273537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1299/2604] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:01.284698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:01.286909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:01.360196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84b698-7f2a-4986-90f3-79ef78be3674 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:01.364388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:01.365158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:01.365483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:01.365756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:01.376590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:01.378063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:01.389843 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1306/2605] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:06.173940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.174491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.174491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.239787 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] MFA Rules not processed for user `3f41efe7f64f41698b5f9ce002c7da71`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:06.241074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:06.323516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:06.368176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1300/2606] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:06.373761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:06.387100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a799ef-1899-4472-9a1e-d81bf7693deb service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:06.396275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:06.397143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.397459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.397863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.398731 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:06.401184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:06.442849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1307/2607] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:06 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:06.447247 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:06.463735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f6bdc-33d9-4478-b09c-37a9c47c2659 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:06.547026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:06.547783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.548033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.548306 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.549057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:06.551242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:06.576402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1301/2608] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:06 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:06.582040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:06.597786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362e252-910f-4f70-bc0e-179ed9427f7f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:06.603142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:06.604008 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.604105 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.604360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.605119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:06.606826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:06.623162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1308/2609] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:06.624416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:06.637407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8758-7201-4cb3-864b-31603384e83a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:06.643284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:06.644194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.644473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.644743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] PATH_INFO: `/v3/projects/81c5c7d75b764ae096ce20b8cc5bf12d/users/24e65a8a92e3472fb5b043361899d9e6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.663114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c5c7d75b764ae096ce20b8cc5bf12d, user_id=24e65a8a92e3472fb5b043361899d9e6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:06.666000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:06.684115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1302/2610] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:06 2022] PUT /identity/v3/projects/81c5c7d75b764ae096ce20b8cc5bf12d/users/24e65a8a92e3472fb5b043361899d9e6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:06.688380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:06.699510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4dfc7-2c6d-42a2-8155-64232f5c34c8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:06.767133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27579d1a-1198-4173-af82-82102936c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:06.767860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27579d1a-1198-4173-af82-82102936c1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.768143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27579d1a-1198-4173-af82-82102936c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.768418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27579d1a-1198-4173-af82-82102936c1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.769248 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27579d1a-1198-4173-af82-82102936c1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:06.770994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27579d1a-1198-4173-af82-82102936c1c9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:06.785326 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1309/2611] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:06.787014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:06.798371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46966-17ea-48e3-b923-5a25467877f1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:06.806183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:06.807212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:06.807513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:06.807793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] PATH_INFO: `/v3/projects/81c5c7d75b764ae096ce20b8cc5bf12d/users/24e65a8a92e3472fb5b043361899d9e6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:06.809848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c5c7d75b764ae096ce20b8cc5bf12d, user_id=24e65a8a92e3472fb5b043361899d9e6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:06.812603 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:06.824127 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1303/2612] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:06 2022] PUT /identity/v3/projects/81c5c7d75b764ae096ce20b8cc5bf12d/users/24e65a8a92e3472fb5b043361899d9e6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:12.379342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6493e6d-e45f-42ee-a832-aaace615309c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:12.379950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6493e6d-e45f-42ee-a832-aaace615309c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:12.380384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6493e6d-e45f-42ee-a832-aaace615309c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:12.425377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c6493e6d-e45f-42ee-a832-aaace615309c None None] MFA Rules not processed for user `24e65a8a92e3472fb5b043361899d9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:12.425377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c6493e6d-e45f-42ee-a832-aaace615309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:12.454505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:12.508315 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1310/2613] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:12.521149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:12.523141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:12.590498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715224d0-689d-477b-b18b-dc56787f6670 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:12.594211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '175dd1ccd6104b18aea8e69bb5ec036a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:12.594899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:12.595204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:12.595502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:12.642608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:12.661904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:12.663388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:12.674519 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1304/2614] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:12.798384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:12.798684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:12.799038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:12.865840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:12.867161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:12.935050 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1311/2615] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:12.987639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:12.989085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:13.094475 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac975d-6e1d-4a49-8147-5ccba0179c3d service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:13.100070 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515eabf-b3a5-41b6-aa92-d6a4ea1029fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:13.100758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515eabf-b3a5-41b6-aa92-d6a4ea1029fe service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:13.101035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515eabf-b3a5-41b6-aa92-d6a4ea1029fe service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:13.101304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515eabf-b3a5-41b6-aa92-d6a4ea1029fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:13.110834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515eabf-b3a5-41b6-aa92-d6a4ea1029fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:13.112415 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515eabf-b3a5-41b6-aa92-d6a4ea1029fe service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:13.120360 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1305/2616] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 15:19:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:19:19.836424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:19.841346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:19.969902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbecfa-6355-4df4-872a-1e110f3b4b54 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:19.974667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:19.975609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:19.975890 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:19.976161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] PATH_INFO: `/v3/users/e53a2143327b4c418cabf6675f64f8a3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:19.977397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53a2143327b4c418cabf6675f64f8a3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:19.979858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:20.027319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:20.051920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53a2143327b4c418cabf6675f64f8a3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.052591 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53a2143327b4c418cabf6675f64f8a3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.053184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53a2143327b4c418cabf6675f64f8a3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.071347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53a2143327b4c418cabf6675f64f8a3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.085567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53a2143327b4c418cabf6675f64f8a3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.112709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:20.112709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:20.113431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:20.114703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1312/2617] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:19 2022] DELETE /identity/v3/users/e53a2143327b4c418cabf6675f64f8a3 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:20.133149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:20.134382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:20.234320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1306/2618] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:20.241959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:20.243785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:20.325337 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244f481-1fa3-46b2-ba95-9928d2fc6800 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:20.329365 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:20.330144 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:20.330537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:20.330937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:20.347445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:20.349089 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:20.360508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1313/2619] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:20.692866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:20.696371 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:20.740491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9b19b-0fe4-475c-8059-b27a5a5ebb6c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:20.745651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:20.746534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:20.746871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:20.747147 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] PATH_INFO: `/v3/projects/362682a090514b81a24d2246460f6993` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:20.748359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] RBAC: Authorizing `identity:delete_project(project_id=362682a090514b81a24d2246460f6993)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:20.750284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:20.814590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362682a090514b81a24d2246460f6993', 'request_id': 'req-cd7593a5-74f5-493e-8546-4d174255643b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.818597 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:20.826721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362682a090514b81a24d2246460f6993', 'request_id': 'req-cd7593a5-74f5-493e-8546-4d174255643b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.835535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362682a090514b81a24d2246460f6993', 'request_id': 'req-cd7593a5-74f5-493e-8546-4d174255643b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.859934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] The token cache is being invalidate because project 362682a090514b81a24d2246460f6993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:20.860377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.861159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:20.868633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1307/2620] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:20 2022] DELETE /identity/v3/projects/362682a090514b81a24d2246460f6993 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:20.871536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:20.882184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1ba4c-583b-43f5-937a-0096d30241b5 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:20.962301 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:20.963227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:20.963505 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:20.963775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] PATH_INFO: `/v3/users/4a89470f01bc442eaa01ec7629b2471d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:20.965020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a89470f01bc442eaa01ec7629b2471d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:20.967039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:21.034086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a89470f01bc442eaa01ec7629b2471d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.034717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a89470f01bc442eaa01ec7629b2471d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.035418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a89470f01bc442eaa01ec7629b2471d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.038392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:21.068255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a89470f01bc442eaa01ec7629b2471d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.077920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6462eaa6-4db8-43d5-b2d1-a4571895a436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a89470f01bc442eaa01ec7629b2471d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.103122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1314/2621] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:20 2022] DELETE /identity/v3/users/4a89470f01bc442eaa01ec7629b2471d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:21.475370 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:21.476936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:21.495977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7593a5-74f5-493e-8546-4d174255643b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:21.565340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:21.566207 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:21.566497 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:21.566786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] PATH_INFO: `/v3/projects/3dadd1e58ecc4e629fbd90fc990a8e24` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:21.568187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dadd1e58ecc4e629fbd90fc990a8e24)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:21.570708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:21.648627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dadd1e58ecc4e629fbd90fc990a8e24', 'request_id': 'req-f008bac0-8ab4-4f87-bf58-88341cb1d20f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.657706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dadd1e58ecc4e629fbd90fc990a8e24', 'request_id': 'req-f008bac0-8ab4-4f87-bf58-88341cb1d20f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.667051 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dadd1e58ecc4e629fbd90fc990a8e24', 'request_id': 'req-f008bac0-8ab4-4f87-bf58-88341cb1d20f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.689520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] The token cache is being invalidate because project 3dadd1e58ecc4e629fbd90fc990a8e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:21.696174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.696174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f008bac0-8ab4-4f87-bf58-88341cb1d20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:21.696174 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1308/2622] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:21 2022] DELETE /identity/v3/projects/3dadd1e58ecc4e629fbd90fc990a8e24 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:21.697771 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:21.698094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:21.706827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:21.719768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:21.721131 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:21.811102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1315/2623] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:21.823924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:21.824431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:21.824772 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:21.844486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:21.845650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:21.861467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1309/2624] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:21.867064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:21.876252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c6233-b0fb-4edd-a0d9-738c16694034 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:21.881405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:21.882086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:21.882360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:21.882628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:21.885184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:19:21.885184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:21.885498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:21.900196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1316/2625] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:19:21.905979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:21.919457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e376572-8a26-4008-94f9-28027796f2f8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:21.924942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:21.925631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:21.925911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:21.926217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:21.931835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:21.934823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.000073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1310/2626] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:21 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:22.004122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.016571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0d0db-0bd7-43c1-897a-1a503869bd21 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.102316 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.103776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.104135 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.104481 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.105253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.107266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.127026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.132485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:22.136242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1317/2627] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:22 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:22.140066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.158738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873817d-754f-427a-8300-66a290dc3a7e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.166102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.167116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.167462 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.167804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.168598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.170283 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.184367 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1318/2628] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:19:22.185419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.201548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358ad6-8e07-41ff-b831-aa33382ecb8b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.203243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:22.208004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.208975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.209487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.209935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.211000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.213188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.218942 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0814d-6d46-419f-bc58-23c9273ffa34 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.224753 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.225649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.226158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.226626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.228527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1319/2629] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:22.229015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:22.232713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.244802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15998a52-678b-4779-8c94-844e5c4a93c2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.250451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd31020-4ae9-4102-9e50-1b990ee4e28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.251611 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd31020-4ae9-4102-9e50-1b990ee4e28f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.251891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd31020-4ae9-4102-9e50-1b990ee4e28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.252161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd31020-4ae9-4102-9e50-1b990ee4e28f admin admin] PATH_INFO: `/v3/projects/f9e64ebccc2f47518569bddf631032db/users/5369285bc6004182b8d49d71c45c4ae2/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.270009 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd31020-4ae9-4102-9e50-1b990ee4e28f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e64ebccc2f47518569bddf631032db, user_id=5369285bc6004182b8d49d71c45c4ae2, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.272933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd31020-4ae9-4102-9e50-1b990ee4e28f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.300232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1320/2630] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:22 2022] PUT /identity/v3/projects/f9e64ebccc2f47518569bddf631032db/users/5369285bc6004182b8d49d71c45c4ae2/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:22.301938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:22.314891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327922e8-f6aa-4dc1-b97d-d828a3713c58 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.315495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327922e8-f6aa-4dc1-b97d-d828a3713c58 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.316236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327922e8-f6aa-4dc1-b97d-d828a3713c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.355208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.356428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-327922e8-f6aa-4dc1-b97d-d828a3713c58 None None] MFA Rules not processed for user `5369285bc6004182b8d49d71c45c4ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:22.356613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.358661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-327922e8-f6aa-4dc1-b97d-d828a3713c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:22.380720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1311/2631] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:22.470961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1321/2632] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:22.481304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.492788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f707223-f646-41da-bf91-24ccb580b34d service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.527536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.527930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.528228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.551017 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] MFA Rules not processed for user `3f41efe7f64f41698b5f9ce002c7da71`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:22.552314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:22.556054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.556727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.557000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.557268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.557995 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.559962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.595955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1312/2633] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:22 2022] POST /identity/v3/projects => generated 393 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:22.600257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.616249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1322/2634] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:22.669358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62b7ed-e397-452c-b4cb-93fa24d8c9bc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.678709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.679361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.679640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.679912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.680606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.682548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.692868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:22.706372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1313/2635] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:22 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:19:22.711978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.723569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b78bc-7052-468d-a0c6-c83b9b64229b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.731933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.739905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.740198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.740480 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.741272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.742985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.762039 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1323/2636] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:22.763308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.773564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae9a272-0d50-4790-8475-69471e3f0db0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.778402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.779342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.779618 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.779888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] PATH_INFO: `/v3/projects/a3e1556264a741ba8abb04229fb5e02e/users/c243e630445c4373a57094f426570eef/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.803613 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e1556264a741ba8abb04229fb5e02e, user_id=c243e630445c4373a57094f426570eef, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.806275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.824925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1314/2637] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:22 2022] PUT /identity/v3/projects/a3e1556264a741ba8abb04229fb5e02e/users/c243e630445c4373a57094f426570eef/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:22.829005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.840188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e41abe-5b03-4e95-a1fa-5f8c333266e0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.904688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c498e19-082c-4fab-a425-f70e94a33fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.905417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c498e19-082c-4fab-a425-f70e94a33fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.905632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c498e19-082c-4fab-a425-f70e94a33fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.905927 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c498e19-082c-4fab-a425-f70e94a33fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.906653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c498e19-082c-4fab-a425-f70e94a33fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.908298 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c498e19-082c-4fab-a425-f70e94a33fdc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.923369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1324/2638] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:19:22.925547 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:22.940975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4ae10-be3b-4343-9e60-46d59a5c80f3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:22.945757 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:22.946640 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.946955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.947220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] PATH_INFO: `/v3/projects/a3e1556264a741ba8abb04229fb5e02e/users/c243e630445c4373a57094f426570eef/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:22.949421 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e1556264a741ba8abb04229fb5e02e, user_id=c243e630445c4373a57094f426570eef, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:22.952612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:22.964588 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1315/2639] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:22 2022] PUT /identity/v3/projects/a3e1556264a741ba8abb04229fb5e02e/users/c243e630445c4373a57094f426570eef/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:19:22.975257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:22.975548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:22.975810 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:23.003928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] MFA Rules not processed for user `c243e630445c4373a57094f426570eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:23.005520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:23.011784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:23.084058 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1325/2640] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:23.095674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:23.103917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:23.166363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1f29c-9a65-4182-8e73-43866f07eb9f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:23.172407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:23.173583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:23.174152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:23.174604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:23.187343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:23.189596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:23.202058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1316/2641] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:23.241637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:23.243054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:23.266349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:23.275693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1056ab0-7073-4a67-a928-c5844d094751 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:23.281474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e46962-c6bc-4d5a-9bef-5090f9e9c0fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:23.282571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e46962-c6bc-4d5a-9bef-5090f9e9c0fe service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:23.282932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e46962-c6bc-4d5a-9bef-5090f9e9c0fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:23.283274 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e46962-c6bc-4d5a-9bef-5090f9e9c0fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:23.337096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac37b5-6b4b-4c04-9726-5b42b9df1a52 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:23.342411 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:23.343097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:23.343375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:23.343637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:23.345533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:23.356782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e46962-c6bc-4d5a-9bef-5090f9e9c0fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:23.357947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:23.358438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e46962-c6bc-4d5a-9bef-5090f9e9c0fe service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:23.359599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:23.368679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1317/2642] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:23.371062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1326/2643] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:24.950880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:24.951458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:24.951458 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:24.971878 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:24.973185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:25.044619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1318/2644] 10.222.0.32 () {58 vars in 1009 bytes} [Wed May 25 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:25.058088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:25.060294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:25.140421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9b997-f36f-4f53-9717-f8773d019d27 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:25.146712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda04198-e41e-4031-82cc-753e7cfbe836 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:25.147537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04198-e41e-4031-82cc-753e7cfbe836 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:25.147821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04198-e41e-4031-82cc-753e7cfbe836 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:25.148099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda04198-e41e-4031-82cc-753e7cfbe836 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:25.157891 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda04198-e41e-4031-82cc-753e7cfbe836 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:25.159556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda04198-e41e-4031-82cc-753e7cfbe836 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:25.170140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1327/2645] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 15:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:19:28.160936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:28.165687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:28.239301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9d84f-9c77-4bcf-b0a6-cdd5d55659ee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:28.244775 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:28.245744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:28.246023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:28.246292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] PATH_INFO: `/v3/users/5369285bc6004182b8d49d71c45c4ae2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:28.247531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5369285bc6004182b8d49d71c45c4ae2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:28.249546 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:28.318023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5369285bc6004182b8d49d71c45c4ae2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:28.324232 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5369285bc6004182b8d49d71c45c4ae2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:28.335563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5369285bc6004182b8d49d71c45c4ae2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:28.337152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5369285bc6004182b8d49d71c45c4ae2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:28.353138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5369285bc6004182b8d49d71c45c4ae2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:28.384329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1319/2646] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:28 2022] DELETE /identity/v3/users/5369285bc6004182b8d49d71c45c4ae2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:28.386468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:28.387417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:28.387847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:28.407035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:28.408180 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:28.427947 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:28.479782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1328/2647] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:28.488077 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:28.489743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:28.582105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bc612-5539-4d81-a87a-088d91aba83e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:28.588985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:28.589741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:28.590018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:28.590286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:28.603225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:28.605245 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:28.622033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1320/2648] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:29.048221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:29.051729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:29.075623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5931890-dd59-4a23-9bad-8df697bed344 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:29.090475 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:29.092052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:29.092368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:29.092653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] PATH_INFO: `/v3/projects/f9e64ebccc2f47518569bddf631032db` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:29.094145 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e64ebccc2f47518569bddf631032db)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:29.115806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:29.224551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e64ebccc2f47518569bddf631032db', 'request_id': 'req-1d5777ec-43b4-4332-b6f6-5533f47b0b06'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.233302 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e64ebccc2f47518569bddf631032db', 'request_id': 'req-1d5777ec-43b4-4332-b6f6-5533f47b0b06'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.241416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e64ebccc2f47518569bddf631032db', 'request_id': 'req-1d5777ec-43b4-4332-b6f6-5533f47b0b06'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.259792 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] The token cache is being invalidate because project f9e64ebccc2f47518569bddf631032db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:29.260285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.261495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.269536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1329/2649] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:29 2022] DELETE /identity/v3/projects/f9e64ebccc2f47518569bddf631032db => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:29.273607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:29.285978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b3c3b-6829-4597-9567-3e098375095a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:29.322970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:29.449593 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:29.451414 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:29.452019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:29.453009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] PATH_INFO: `/v3/users/c243e630445c4373a57094f426570eef` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:29.455050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c243e630445c4373a57094f426570eef)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:29.457576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:29.536443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c243e630445c4373a57094f426570eef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.537065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c243e630445c4373a57094f426570eef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.545557 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c243e630445c4373a57094f426570eef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.546280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c243e630445c4373a57094f426570eef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.566256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-52da7c1a-b874-49b4-8bce-33f68803a837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c243e630445c4373a57094f426570eef'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:29.584268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1321/2650] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:29 2022] DELETE /identity/v3/users/c243e630445c4373a57094f426570eef => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:29.938884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:29.940540 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:29.957098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5777ec-43b4-4332-b6f6-5533f47b0b06 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.054800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.054800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.054800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.054800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] PATH_INFO: `/v3/projects/a3e1556264a741ba8abb04229fb5e02e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.055514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e1556264a741ba8abb04229fb5e02e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.055514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.117930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:30.120372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e1556264a741ba8abb04229fb5e02e', 'request_id': 'req-669cd019-f4a5-4457-8a16-51bf542664bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.129170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e1556264a741ba8abb04229fb5e02e', 'request_id': 'req-669cd019-f4a5-4457-8a16-51bf542664bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.140374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e1556264a741ba8abb04229fb5e02e', 'request_id': 'req-669cd019-f4a5-4457-8a16-51bf542664bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.161335 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] The token cache is being invalidate because project a3e1556264a741ba8abb04229fb5e02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:30.161766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.162451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-669cd019-f4a5-4457-8a16-51bf542664bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.164407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1330/2651] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:29 2022] DELETE /identity/v3/projects/a3e1556264a741ba8abb04229fb5e02e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:30.169244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.169764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.170040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.194304 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:30.195569 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.259143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1322/2652] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:30.268175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.268461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.268735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.287787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:30.289185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.298331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1331/2653] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:30.302570 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.313140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef0c8a-aa07-4cbb-8df9-a8af1f5bfc5a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.318109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.318776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.319151 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.319448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.320183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:19:30.320538 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.322160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.338834 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1323/2654] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:19:30.341319 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.353304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe5ed5-8e0b-47fe-980e-cdc76df0ed21 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.358110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.358879 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.359198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.359491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.360187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.362206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.390850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1332/2655] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:30 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:30.394931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.421368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7066729-aeec-4d04-b416-c14d09c18332 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.455509 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:30.466814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.469597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.550033 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.550837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.551246 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.551420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.552201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.554210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.578397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac05cc-e26c-45ad-a251-d4155fc0baf5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.586133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.589495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.589495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.589495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.603056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.607276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1324/2656] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:30 2022] POST /identity/v3/users => generated 301 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:30.608361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.610112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.629104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc937f61-d310-4356-96cd-91d4bfc443f5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.640743 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.642086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.642417 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.642712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] PATH_INFO: `/v3/users/12888093e02c4d6f91326e261218698c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.644612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=12888093e02c4d6f91326e261218698c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.647524 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.756748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12888093e02c4d6f91326e261218698c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.758109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12888093e02c4d6f91326e261218698c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.771582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12888093e02c4d6f91326e261218698c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.771830 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.772348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12888093e02c4d6f91326e261218698c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.773599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.798791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ff5e186-b935-41c5-a194-1981c5dd9b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12888093e02c4d6f91326e261218698c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:30.801018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1333/2657] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:30.802597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.820654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f873f-bb62-4c95-b09e-1f32f7e61dc0 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.829567 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.830859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.831351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.831865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.833077 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.833814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1325/2658] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:30 2022] DELETE /identity/v3/users/12888093e02c4d6f91326e261218698c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:30.835977 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:30.839829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.840298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.840423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.858231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1334/2659] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:30.872755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:30.879882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.889560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:30.890877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:30.903762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330debfa-8510-4aa7-8ce1-0d3ad641a815 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:30.942698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:30.984070 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:30.984898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:30.985209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:30.985550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:30.986449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:30.988372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.003033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1335/2660] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:31.003564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1326/2661] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:31.006592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.015785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.016569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96518d-b25b-4f85-9b5a-37503b538596 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.017187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:31.029524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f3d46-a9ab-4895-a6de-ca5179a16cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.030451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f3d46-a9ab-4895-a6de-ca5179a16cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.030846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f3d46-a9ab-4895-a6de-ca5179a16cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.031197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f3d46-a9ab-4895-a6de-ca5179a16cae admin admin] PATH_INFO: `/v3/projects/e5826f3b7a144f73ba316e1ae5bcb3c9/users/140e8fae83544b928831412f74c55b61/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.060842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:31.061046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f3d46-a9ab-4895-a6de-ca5179a16cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5826f3b7a144f73ba316e1ae5bcb3c9, user_id=140e8fae83544b928831412f74c55b61, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.063563 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f3d46-a9ab-4895-a6de-ca5179a16cae admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.106483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1336/2662] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:31 2022] PUT /identity/v3/projects/e5826f3b7a144f73ba316e1ae5bcb3c9/users/140e8fae83544b928831412f74c55b61/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:31.117403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.117736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.118048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.120661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d9dd3-c105-4bc3-a77c-bc1efaa8e19e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.190446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] MFA Rules not processed for user `140e8fae83544b928831412f74c55b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:31.193097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:31.208123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.209057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.209343 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.209731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.299388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.301030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.312973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1327/2663] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:31.327106 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1337/2664] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:31.332183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.344462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fa2b-627c-487f-960d-2b8fbd705010 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.354106 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.354868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.355155 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.355434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.356925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.359016 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.410837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1328/2665] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:31 2022] POST /identity/v3/projects => generated 401 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:31.411075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.428408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cee7f-3588-4439-9608-130bf32feb52 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.535378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.536313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.536498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.536777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.537678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.543662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.582151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1338/2666] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:31 2022] POST /identity/v3/users => generated 307 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:19:31.586552 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.601981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9e880-f238-46cc-80a1-93b4ed5c71f2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.616311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.616986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.617285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.617545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.618289 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.629876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.647747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1329/2667] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:31.651919 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.663198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5e019-0cea-40a9-b960-3fadaf09aa39 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.668873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.670296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.671038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.671539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] PATH_INFO: `/v3/projects/14c8a30e11fa42ca899b62e85542ee7a/users/e029028524df40cb9ea73e8202718cda/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.696257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c8a30e11fa42ca899b62e85542ee7a, user_id=e029028524df40cb9ea73e8202718cda, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.701143 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.724141 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1339/2668] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:31 2022] PUT /identity/v3/projects/14c8a30e11fa42ca899b62e85542ee7a/users/e029028524df40cb9ea73e8202718cda/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:31.727935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.740128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a33ab-8e6b-413f-b4cb-74a7d430c733 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.760064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.762991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:31.772417 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:31.868746 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e0c25-9a25-4949-8926-90d095697cd3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.874379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.875473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.875884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.876261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] PATH_INFO: `/v3/projects/74aae67b92a142d68bc7302a97f7f84b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.877652 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74aae67b92a142d68bc7302a97f7f84b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.879898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.898988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:31.908010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:31.909970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:31.914823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:31.915188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:31.916316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:31.918307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:31.937567 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1330/2669] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:31.941555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:31.948941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74aae67b92a142d68bc7302a97f7f84b', 'request_id': 'req-91b1a9d3-7213-40dc-837c-4ab391610394'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:31.954425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633fcbd-2c72-4092-b57b-32b15ce36980 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:31.956139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74aae67b92a142d68bc7302a97f7f84b', 'request_id': 'req-91b1a9d3-7213-40dc-837c-4ab391610394'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:31.962847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74aae67b92a142d68bc7302a97f7f84b', 'request_id': 'req-91b1a9d3-7213-40dc-837c-4ab391610394'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:31.999422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] The token cache is being invalidate because project 74aae67b92a142d68bc7302a97f7f84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:31.999864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:32.001671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91b1a9d3-7213-40dc-837c-4ab391610394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:32.005456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1340/2670] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:31 2022] DELETE /identity/v3/projects/74aae67b92a142d68bc7302a97f7f84b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:19:32.012635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af147f-0e27-479a-beda-e349cd4cda84 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.014920 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af147f-0e27-479a-beda-e349cd4cda84 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.015408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af147f-0e27-479a-beda-e349cd4cda84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.060963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-04af147f-0e27-479a-beda-e349cd4cda84 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:32.064259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04af147f-0e27-479a-beda-e349cd4cda84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.080875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.081886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.082166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.082440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] PATH_INFO: `/v3/projects/14c8a30e11fa42ca899b62e85542ee7a/users/e029028524df40cb9ea73e8202718cda/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.086742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c8a30e11fa42ca899b62e85542ee7a, user_id=e029028524df40cb9ea73e8202718cda, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.089520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.100857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1331/2671] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:31 2022] PUT /identity/v3/projects/14c8a30e11fa42ca899b62e85542ee7a/users/e029028524df40cb9ea73e8202718cda/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:19:32.113545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.116376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.185190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1341/2672] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:32.186612 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7581117-b121-49a5-865f-cb904b64b92e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.187840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7581117-b121-49a5-865f-cb904b64b92e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.188123 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7581117-b121-49a5-865f-cb904b64b92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.215792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0bfe3-e190-4b92-b5f3-dc24a5a6e4d1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.227727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.231454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.232941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.233426 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.236267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.241651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f7581117-b121-49a5-865f-cb904b64b92e None None] MFA Rules not processed for user `e029028524df40cb9ea73e8202718cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:32.243185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f7581117-b121-49a5-865f-cb904b64b92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.276099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:32.338794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1342/2673] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:32.340167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.340167 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.340358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.353321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.354790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.360953 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:32.362448 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.367658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1332/2674] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:32.383838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.385584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.453359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:32.482159 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1343/2675] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:32.486987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.494873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f71d-60a5-4973-bc08-fed4d1586b32 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.498598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.499051 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563464f-0276-408b-a3d1-1276411ada0b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.499296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.499571 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.499840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.504834 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.505741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.506179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.506596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.512857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.514338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.522006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:19:32.522846 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.525739 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.526657 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1333/2676] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:32.564643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1344/2677] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:19:32.570693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.593177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e0998c-dc46-4b09-b51d-b7c67c1fa714 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.599703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.602134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.602134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.602134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.602134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.604686 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.649249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1334/2678] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:32 2022] POST /identity/v3/projects => generated 361 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:32.652955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.659443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.664431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e86a8f-1592-40b4-9670-d1432b5f1603 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.721428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ddff4-9130-4399-abd8-b298fd1fca80 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.725243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.725958 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.726243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.726510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.728688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:32.746814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.747551 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.748388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.748690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.749822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.751954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.790450 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1345/2679] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:32 2022] POST /identity/v3/users => generated 281 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:32.839090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.841582 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.852848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.856334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1335/2680] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:19:32.867569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c993fd-5249-4e4f-bce9-6aea3c657bd6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.873537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.874198 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.874484 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.874848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.875515 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.888470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.900838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1346/2681] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:19:32.905023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.917013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aff93-ebd9-4fe6-b8e8-88f8b7cc2f73 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.922685 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ab9d5-0bff-4aa7-b495-ddaa9f6edbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.923483 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ab9d5-0bff-4aa7-b495-ddaa9f6edbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.923847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ab9d5-0bff-4aa7-b495-ddaa9f6edbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.924190 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ab9d5-0bff-4aa7-b495-ddaa9f6edbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:32.925011 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ab9d5-0bff-4aa7-b495-ddaa9f6edbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:32.926882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ab9d5-0bff-4aa7-b495-ddaa9f6edbd3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:32.943409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:32.950316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1336/2682] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:32.956714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea5298-390b-4137-8f5f-1b17a63e6155 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:32.967821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:32.970614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:32.971311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:32.972469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] PATH_INFO: `/v3/projects/30cee761abd84bebb2519619c0b16e58/users/624cb3f90a454233a26f7d6f05fee456/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:33.010422 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] RBAC: Authorizing `identity:create_grant(project_id=30cee761abd84bebb2519619c0b16e58, user_id=624cb3f90a454233a26f7d6f05fee456, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:33.013539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:33.038827 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1347/2683] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:32 2022] PUT /identity/v3/projects/30cee761abd84bebb2519619c0b16e58/users/624cb3f90a454233a26f7d6f05fee456/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:33.042184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:33.042507 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:33.042847 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:33.082545 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] MFA Rules not processed for user `624cb3f90a454233a26f7d6f05fee456`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:33.084238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:33.131079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:33.162733 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1337/2684] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:33.197049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:33.198894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:33.204350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:33.285361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f5634-922a-4f9e-8c78-9cea50a6583c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:33.289308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c035b6d-2edc-411a-93bd-494d053485db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:33.290338 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c035b6d-2edc-411a-93bd-494d053485db service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:33.290968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c035b6d-2edc-411a-93bd-494d053485db service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:33.291756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c035b6d-2edc-411a-93bd-494d053485db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:33.303268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c035b6d-2edc-411a-93bd-494d053485db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:33.304714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c035b6d-2edc-411a-93bd-494d053485db service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:33.314509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1348/2685] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:33.330111 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:33.394583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d27cd6-b5d9-452d-a3d7-38368fefc3a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:33.399566 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:33.400428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:33.400703 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:33.400969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] PATH_INFO: `/v3/users/624cb3f90a454233a26f7d6f05fee456` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:33.402165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624cb3f90a454233a26f7d6f05fee456)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:33.404164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:33.454057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624cb3f90a454233a26f7d6f05fee456'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:33.454908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624cb3f90a454233a26f7d6f05fee456'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:33.462989 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624cb3f90a454233a26f7d6f05fee456'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:33.463771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624cb3f90a454233a26f7d6f05fee456'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:33.476653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624cb3f90a454233a26f7d6f05fee456'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:33.491095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1338/2686] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:33 2022] DELETE /identity/v3/users/624cb3f90a454233a26f7d6f05fee456 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:33.495944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:33.496294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:33.496648 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:33.515730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:33.516970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:33.584195 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1349/2687] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:33.594587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:33.596436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:33.658301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9f5b5-aaf1-4d67-b45c-febf11049d03 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:33.662291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadd401-b4f2-4c51-ab55-84c5450a51a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:33.663010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadd401-b4f2-4c51-ab55-84c5450a51a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:33.663334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadd401-b4f2-4c51-ab55-84c5450a51a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:33.663666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadd401-b4f2-4c51-ab55-84c5450a51a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:33.673687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadd401-b4f2-4c51-ab55-84c5450a51a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:33.675216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadd401-b4f2-4c51-ab55-84c5450a51a7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:33.685577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1339/2688] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:33.990357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:33.992425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:34.005250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab0059-8f41-4dc2-96d9-8b712229b9fd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.010281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.011240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.011669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.011941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] PATH_INFO: `/v3/projects/30cee761abd84bebb2519619c0b16e58` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.013253 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=30cee761abd84bebb2519619c0b16e58)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.015246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.068483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30cee761abd84bebb2519619c0b16e58', 'request_id': 'req-5602392b-d0f3-429e-9a8b-74ca7071a14c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:34.074877 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30cee761abd84bebb2519619c0b16e58', 'request_id': 'req-5602392b-d0f3-429e-9a8b-74ca7071a14c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:34.080572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30cee761abd84bebb2519619c0b16e58', 'request_id': 'req-5602392b-d0f3-429e-9a8b-74ca7071a14c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:34.100033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] The token cache is being invalidate because project 30cee761abd84bebb2519619c0b16e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:34.100377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:34.101415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5602392b-d0f3-429e-9a8b-74ca7071a14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:34.103943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1350/2689] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:33 2022] DELETE /identity/v3/projects/30cee761abd84bebb2519619c0b16e58 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:34.112517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.113003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.113471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.120609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:34.135153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:34.136393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:34.201986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1340/2690] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:34.210023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.210308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.210584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.232062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:34.233378 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:34.242130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1351/2691] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:34.247133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.260128 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0844950-cd4f-422b-bd90-0772334f94b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.265660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.266291 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.266725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.267031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.267761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:19:34.268067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.269701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.283591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1341/2692] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:19:34.288696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.299471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17fcfd-7866-4009-a796-d8847a50761b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.304721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.305479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.305782 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.306044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.306736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.309010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.333355 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:34.344765 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1352/2693] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:34 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:34.345915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.356093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae60d3-2795-428b-9e9d-a1b1f52bf007 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.424884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.425987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.426645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.427093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.428339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.431442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.458982 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1342/2694] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:34 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:19:34.463461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.474745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3ea3f-eb59-4446-a096-c1d46a862a9d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.480950 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.481625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.481904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.482174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.482942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.484630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.497572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1353/2695] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:34.501873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.513076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591f10e-725f-4528-995e-df9f0fedc5f3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.519064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3d620-838b-4fd2-a102-2fa17108cd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.519722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d620-838b-4fd2-a102-2fa17108cd82 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.519997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d620-838b-4fd2-a102-2fa17108cd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.520349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d620-838b-4fd2-a102-2fa17108cd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.521071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3d620-838b-4fd2-a102-2fa17108cd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.522945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3d620-838b-4fd2-a102-2fa17108cd82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.535561 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1343/2696] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:34.539908 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.552656 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008634fd-59d5-460c-8cb6-282d2c527969 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.558339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.559309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.559583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.559847 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] PATH_INFO: `/v3/projects/68187b54c7d448ddb0ef2cb13d0cd92d/users/2163d4a3692b4ab3b7a4cc7efe6b9310/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.575576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68187b54c7d448ddb0ef2cb13d0cd92d, user_id=2163d4a3692b4ab3b7a4cc7efe6b9310, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.578245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.592424 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1354/2697] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:34 2022] PUT /identity/v3/projects/68187b54c7d448ddb0ef2cb13d0cd92d/users/2163d4a3692b4ab3b7a4cc7efe6b9310/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:34.604076 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.604381 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.604673 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.631230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] MFA Rules not processed for user `2163d4a3692b4ab3b7a4cc7efe6b9310`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:34.632531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:34.696721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:34.702864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1344/2698] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:34.714186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:34.716064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:34.777620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852c1e5-9842-401d-9c1d-2a1878efeb0c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:34.781471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:34.782133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:34.783878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:34.783878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:34.792901 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:34.794717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:34.805619 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1355/2699] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:44.704225 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:44.801216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:44.918959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:44.920602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:44.982276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b90c26-edaf-42af-a94d-a717dd5b80bc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:44.986040 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:44.986683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:44.986996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:44.987261 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:44.989139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:45.001043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:45.002479 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:45.011328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1345/2700] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:45.295741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:45.297196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:45.381798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259d035-955a-46a3-a475-a1d1ae226b65 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:45.387055 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:45.387732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:45.388012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:45.388288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:45.390733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:45.406035 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:45.407894 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e979b0-0c74-485c-a9f1-3f28f6c65dea service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:45.419076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1356/2701] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:47.618791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:47.620266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:47.694096 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497354d7-ecd7-41c7-b866-2d5655e6eccd service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:47.698835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:47.699674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:47.699943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:47.700205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] PATH_INFO: `/v3/users/140e8fae83544b928831412f74c55b61` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:47.701459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=140e8fae83544b928831412f74c55b61)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:47.703436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:47.766970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140e8fae83544b928831412f74c55b61'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:47.767914 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140e8fae83544b928831412f74c55b61'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:47.777386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140e8fae83544b928831412f74c55b61'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:47.778374 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140e8fae83544b928831412f74c55b61'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:47.802935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140e8fae83544b928831412f74c55b61'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:47.821217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1346/2702] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:47 2022] DELETE /identity/v3/users/140e8fae83544b928831412f74c55b61 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:47.826499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:47.827031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:47.827240 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:47.847224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:47.848604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:47.937194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1357/2703] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:47.947348 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:47.948940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:47.978542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:48.016774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5eba00-a8b3-4ad9-9058-c3baa9de1533 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:48.023808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:48.023808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:48.024125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:48.024407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:48.036399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:48.037844 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:48.048057 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1347/2704] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:48.330443 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:48.332072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:48.349027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c9c26-e545-4f7c-a498-06bfc6072e80 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:48.357412 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:48.358985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:48.359477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:48.360031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] PATH_INFO: `/v3/projects/e5826f3b7a144f73ba316e1ae5bcb3c9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:48.362074 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5826f3b7a144f73ba316e1ae5bcb3c9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:48.364832 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:48.414040 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5826f3b7a144f73ba316e1ae5bcb3c9', 'request_id': 'req-d3f9d210-1997-4bad-abe5-442906e74d0c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.420427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5826f3b7a144f73ba316e1ae5bcb3c9', 'request_id': 'req-d3f9d210-1997-4bad-abe5-442906e74d0c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.426272 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5826f3b7a144f73ba316e1ae5bcb3c9', 'request_id': 'req-d3f9d210-1997-4bad-abe5-442906e74d0c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.440883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] The token cache is being invalidate because project e5826f3b7a144f73ba316e1ae5bcb3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:48.441170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.441806 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.443386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1358/2705] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:48 2022] DELETE /identity/v3/projects/e5826f3b7a144f73ba316e1ae5bcb3c9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:48.448027 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:48.461344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39036ed0-a6d9-42be-9a8b-13112f0a1077 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:48.519399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:48.520264 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:48.520537 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:48.520824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] PATH_INFO: `/v3/users/e029028524df40cb9ea73e8202718cda` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:48.522004 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e029028524df40cb9ea73e8202718cda)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:48.523980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:48.571062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e029028524df40cb9ea73e8202718cda'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.571735 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e029028524df40cb9ea73e8202718cda'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.578542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e029028524df40cb9ea73e8202718cda'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.579201 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e029028524df40cb9ea73e8202718cda'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.592420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0363630-300e-4383-9677-e9153eaaa5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e029028524df40cb9ea73e8202718cda'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:48.607857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1348/2706] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:19:48 2022] DELETE /identity/v3/users/e029028524df40cb9ea73e8202718cda => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:19:48.900354 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:48.901983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:48.917038 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9d210-1997-4bad-abe5-442906e74d0c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:48.928628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:48.981219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:48.982000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:48.982284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:48.982560 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] PATH_INFO: `/v3/projects/14c8a30e11fa42ca899b62e85542ee7a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:48.983790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c8a30e11fa42ca899b62e85542ee7a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:48.985713 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.038549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c8a30e11fa42ca899b62e85542ee7a', 'request_id': 'req-30bc6764-892d-469e-8f8b-e91bcdb1280f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:49.045607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c8a30e11fa42ca899b62e85542ee7a', 'request_id': 'req-30bc6764-892d-469e-8f8b-e91bcdb1280f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:49.052784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c8a30e11fa42ca899b62e85542ee7a', 'request_id': 'req-30bc6764-892d-469e-8f8b-e91bcdb1280f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:49.068387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] The token cache is being invalidate because project 14c8a30e11fa42ca899b62e85542ee7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:19:49.068387 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:49.068661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-30bc6764-892d-469e-8f8b-e91bcdb1280f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:19:49.070492 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1359/2707] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:19:48 2022] DELETE /identity/v3/projects/14c8a30e11fa42ca899b62e85542ee7a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:49.075669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.076058 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.076376 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.102645 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:49.104393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:49.161152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:49.191943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1349/2708] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:49.200404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.200661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.200938 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.221473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:49.222712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:49.230957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1360/2709] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:19:49.236438 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.250146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b0b57-acd6-432f-8812-995a97bc88a9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.256168 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.256838 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.257121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.257411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.258176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:19:49.258472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.260108 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.274139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1350/2710] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:19:49.278882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.289228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b15635-b740-400c-a85d-5351a6fef270 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.295200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.295870 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.296154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.296427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.297103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.299075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.334395 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1361/2711] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:19:49 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:49.336591 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.348997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7186ded-80bb-4f2d-9306-416ec5ab4529 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.418867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.419535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.419818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.420085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.420781 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.422731 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.445410 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1351/2712] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:19:49 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:19:49.449885 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.463238 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8000f7b-9a85-4b01-a56b-d24ee837fec3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.468859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.469588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.469867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.470138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.470904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.472556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.483598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1362/2713] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:19:49.488473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.498708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a182021-4383-4600-b325-20020fb90897 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.504498 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cea652-bf80-4c97-8a53-aa1da0a4b024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.505158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cea652-bf80-4c97-8a53-aa1da0a4b024 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.505431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cea652-bf80-4c97-8a53-aa1da0a4b024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.505695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cea652-bf80-4c97-8a53-aa1da0a4b024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.506418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cea652-bf80-4c97-8a53-aa1da0a4b024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.508099 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cea652-bf80-4c97-8a53-aa1da0a4b024 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.519160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1352/2714] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:19:49.523990 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.534629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07771856-16ff-4f4f-a8f2-4de5587a5d35 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.540030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.540922 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.541201 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.541479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] PATH_INFO: `/v3/projects/01ad03e34ca64c9d847aa0af14e0d707/users/2010cb9f3cef497793ecda1e231dde63/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.558034 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ad03e34ca64c9d847aa0af14e0d707, user_id=2010cb9f3cef497793ecda1e231dde63, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.560763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.577222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1363/2715] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:19:49 2022] PUT /identity/v3/projects/01ad03e34ca64c9d847aa0af14e0d707/users/2010cb9f3cef497793ecda1e231dde63/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:19:49.589339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.589819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.590267 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.628998 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] MFA Rules not processed for user `2010cb9f3cef497793ecda1e231dde63`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:19:49.630227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:49.705331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1353/2716] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:19:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:19:49.717601 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:49.719822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:49.771212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:49.789997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b677b3-f353-475b-8d49-1146a6bb1800 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:49.794012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:49.794717 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:49.795032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:49.795305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:49.809257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:49.810785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:49.821975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1364/2717] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:58.404278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:19:58.405925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:58.452208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:19:58.486039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2d078-060c-450c-a131-94326065b4fc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:19:58.492020 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:19:58.492668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:19:58.492944 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:19:58.493208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:19:58.495134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:19:58.507471 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:19:58.509019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69818dd8-8422-43cd-b6a0-c14ff84b130e service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:19:58.519503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1354/2718] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:19:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:19:59.816667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:00.270325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:00.385512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3bed8-b170-49da-a413-647edc04beb3 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:00.391096 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:00.392053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:00.392307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:00.392564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] PATH_INFO: `/v3/users/2010cb9f3cef497793ecda1e231dde63` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:00.394043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2010cb9f3cef497793ecda1e231dde63)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:00.396237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:00.459350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2010cb9f3cef497793ecda1e231dde63'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:00.460345 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2010cb9f3cef497793ecda1e231dde63'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:00.460969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2010cb9f3cef497793ecda1e231dde63'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:00.473837 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2010cb9f3cef497793ecda1e231dde63'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:00.480917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2010cb9f3cef497793ecda1e231dde63'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:00.498701 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1365/2719] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:20:00 2022] DELETE /identity/v3/users/2010cb9f3cef497793ecda1e231dde63 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:00.503777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:00.504452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:00.504452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:00.523809 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:00.525170 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:00.596441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1355/2720] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:20:00.608408 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:00.609916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:00.674136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f79bb-0f54-4152-b7fc-9ce291b488a9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:00.678245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:00.679041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:00.679325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:00.679593 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:00.689629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:00.691090 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:00.700996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1366/2721] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:01.002701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:01.004344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:01.017189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e9272-7c43-43ea-86e2-2592ae21b45a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:01.022389 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:01.023323 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:01.023600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:01.023867 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] PATH_INFO: `/v3/projects/01ad03e34ca64c9d847aa0af14e0d707` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:01.025173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ad03e34ca64c9d847aa0af14e0d707)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:01.027143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:01.077189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ad03e34ca64c9d847aa0af14e0d707', 'request_id': 'req-f1c44fb4-d73c-4a56-bf53-bd229a867d58'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:01.093062 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ad03e34ca64c9d847aa0af14e0d707', 'request_id': 'req-f1c44fb4-d73c-4a56-bf53-bd229a867d58'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:01.099138 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ad03e34ca64c9d847aa0af14e0d707', 'request_id': 'req-f1c44fb4-d73c-4a56-bf53-bd229a867d58'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:01.113776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] The token cache is being invalidate because project 01ad03e34ca64c9d847aa0af14e0d707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:20:01.114093 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:01.114924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1c44fb4-d73c-4a56-bf53-bd229a867d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:01.116715 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1356/2722] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:20:01 2022] DELETE /identity/v3/projects/01ad03e34ca64c9d847aa0af14e0d707 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:04.675011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:04.676671 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:04.690470 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de3fdf-a7ca-425b-a068-25139e3785fb service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:04.710258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:04.749980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:04.750980 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:04.751262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:04.751533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] PATH_INFO: `/v3/users/2163d4a3692b4ab3b7a4cc7efe6b9310` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:04.752744 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2163d4a3692b4ab3b7a4cc7efe6b9310)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:04.754694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:04.810734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2163d4a3692b4ab3b7a4cc7efe6b9310'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:04.811570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2163d4a3692b4ab3b7a4cc7efe6b9310'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:04.812268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2163d4a3692b4ab3b7a4cc7efe6b9310'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:04.837699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2163d4a3692b4ab3b7a4cc7efe6b9310'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:04.845234 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2163d4a3692b4ab3b7a4cc7efe6b9310'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:04.864975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1367/2723] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:20:04 2022] DELETE /identity/v3/users/2163d4a3692b4ab3b7a4cc7efe6b9310 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:04.870912 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:04.871470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:04.871470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:04.894999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:04.896259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:04.956014 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:04.975576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1357/2724] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:20:04.985858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:04.987507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:05.050158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73002b1-11af-47dd-b718-64790716d64d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.054239 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a89801-33b8-4d48-b5c8-956ce1483f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.054828 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a89801-33b8-4d48-b5c8-956ce1483f19 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.055166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a89801-33b8-4d48-b5c8-956ce1483f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.055428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a89801-33b8-4d48-b5c8-956ce1483f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.065016 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a89801-33b8-4d48-b5c8-956ce1483f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.066449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a89801-33b8-4d48-b5c8-956ce1483f19 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.075783 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1368/2725] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:05.365337 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.366904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:05.381470 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72212a-2b98-41f7-96a7-8ee034cde36e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.386823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.387927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.388222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.388532 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] PATH_INFO: `/v3/projects/68187b54c7d448ddb0ef2cb13d0cd92d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.389741 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68187b54c7d448ddb0ef2cb13d0cd92d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.391773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.439877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68187b54c7d448ddb0ef2cb13d0cd92d', 'request_id': 'req-866825d0-0052-448e-99b6-2b1799f0b579'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:05.446646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68187b54c7d448ddb0ef2cb13d0cd92d', 'request_id': 'req-866825d0-0052-448e-99b6-2b1799f0b579'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:05.452724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68187b54c7d448ddb0ef2cb13d0cd92d', 'request_id': 'req-866825d0-0052-448e-99b6-2b1799f0b579'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:05.467646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] The token cache is being invalidate because project 68187b54c7d448ddb0ef2cb13d0cd92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:20:05.467915 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:05.468584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-866825d0-0052-448e-99b6-2b1799f0b579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:20:05.470204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1358/2726] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:20:05 2022] DELETE /identity/v3/projects/68187b54c7d448ddb0ef2cb13d0cd92d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:05.475426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.475709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.476007 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.495418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:05.496628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:05.539147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:05.562244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1369/2727] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:20:05.570219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.570518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.570852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.599864 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:05.601145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:05.609774 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1359/2728] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:20:05.614061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.624849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462532d-ceae-40ea-ad98-605111b6b37a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.630136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.630826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.631104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.631372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.632098 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:20:05.632403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.634043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.645575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1370/2729] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:20:05.650028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.659338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c94ab-375f-44f0-94d6-9360ecf88ed1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.667669 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.668330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.668609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.668880 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.669576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.671551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.697763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1360/2730] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:20:05 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:20:05.703069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.716848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb3a43-84d7-4503-9225-61fc0714a305 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.778543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.779293 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.779576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.779850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.780553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.782521 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.804966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1371/2731] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:20:05 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:20:05.808981 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.819916 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e1fdf-2682-4017-b422-0842053e8532 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.825279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.825917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.826193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.826463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.827223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.828824 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.839604 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1361/2732] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:20:05.843718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.854532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26dde4-1a98-44f9-ba92-7b62698e6143 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.859926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcfe36-8d4e-4463-869a-a2c28eba1c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.860534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcfe36-8d4e-4463-869a-a2c28eba1c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.860814 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcfe36-8d4e-4463-869a-a2c28eba1c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.861118 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcfe36-8d4e-4463-869a-a2c28eba1c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.861848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcfe36-8d4e-4463-869a-a2c28eba1c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.863514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcfe36-8d4e-4463-869a-a2c28eba1c46 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.874054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1372/2733] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:20:05.878489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:05.888962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922cb0e-0d6f-41aa-9af6-62fc4f73514d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:05.894818 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:05.895702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.895969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.896233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] PATH_INFO: `/v3/projects/88dfcfb36400474aa945f747610c5ad2/users/3db86b8d51074eec9ea8a10cacb8a602/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.911302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dfcfb36400474aa945f747610c5ad2, user_id=3db86b8d51074eec9ea8a10cacb8a602, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:05.913945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:05.930164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1362/2734] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:20:05 2022] PUT /identity/v3/projects/88dfcfb36400474aa945f747610c5ad2/users/3db86b8d51074eec9ea8a10cacb8a602/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:05.935369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52570df-f848-4780-ae22-d78815e2dbff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:05.935663 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52570df-f848-4780-ae22-d78815e2dbff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:05.935942 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52570df-f848-4780-ae22-d78815e2dbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:05.955014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b52570df-f848-4780-ae22-d78815e2dbff None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:05.956261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b52570df-f848-4780-ae22-d78815e2dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:06.030537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1373/2735] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:20:06.040506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:06.042315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:06.100706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63f448-0539-4888-a064-9727aaaafce7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:06.104826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:06.105490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:06.105765 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:06.106030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:06.116951 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:06.118336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:06.130224 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1363/2736] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:10.286266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1184ab-d404-4750-9528-e65795761069 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.286876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1184ab-d404-4750-9528-e65795761069 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.287083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1184ab-d404-4750-9528-e65795761069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.319896 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1a1184ab-d404-4750-9528-e65795761069 None None] MFA Rules not processed for user `3db86b8d51074eec9ea8a10cacb8a602`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:10.321380 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a1184ab-d404-4750-9528-e65795761069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:10.373242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:10.402022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1374/2737] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:20:10.408579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:10.419680 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada4a32-43c8-4414-8319-233500e02b32 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:10.438052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:10.438587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.438908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.439186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.439887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:10.441889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:10.452212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:10.480762 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1184ab-d404-4750-9528-e65795761069 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:10.481758 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1364/2738] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:20:10 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:20:10.497189 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1184ab-d404-4750-9528-e65795761069 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:10.564984 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:10.565693 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.565974 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.566262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.566993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:10.569032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:10.591474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1375/2739] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:20:10 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:20:10.595573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:10.610563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd5a50-db16-4ee8-ab40-97aca35ed187 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:10.618812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:10.618812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.618812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.622911 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.623080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:10.625598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:10.643748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1365/2740] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:20:10.648069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:10.658692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0e5ed-656d-450c-8d03-436a5da4191c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:10.663491 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:10.664685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.665129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.665472 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] PATH_INFO: `/v3/projects/3d2eaffc522042d981e4496050bf71ed/users/05d5004da6b14a26843a55faa346c6cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.684446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2eaffc522042d981e4496050bf71ed, user_id=05d5004da6b14a26843a55faa346c6cd, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:10.687489 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:10.703580 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1376/2741] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:20:10 2022] PUT /identity/v3/projects/3d2eaffc522042d981e4496050bf71ed/users/05d5004da6b14a26843a55faa346c6cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:10.707877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:10.721134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacba1f8-a685-40ec-a190-fb32edb95ed6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:10.783441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:10.784396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.784494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.784702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.785459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:10.790870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:10.803188 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1366/2742] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:20:10.807900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:10.819151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5baeac-6942-491d-84df-e7c77b95f227 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:10.824607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34865c5c-8e7b-4285-abb7-8c7e90249eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:10.825615 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865c5c-8e7b-4285-abb7-8c7e90249eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:10.825976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865c5c-8e7b-4285-abb7-8c7e90249eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:10.826311 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865c5c-8e7b-4285-abb7-8c7e90249eb5 admin admin] PATH_INFO: `/v3/projects/3d2eaffc522042d981e4496050bf71ed/users/05d5004da6b14a26843a55faa346c6cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:10.828528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34865c5c-8e7b-4285-abb7-8c7e90249eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2eaffc522042d981e4496050bf71ed, user_id=05d5004da6b14a26843a55faa346c6cd, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:10.831379 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34865c5c-8e7b-4285-abb7-8c7e90249eb5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:10.843262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1377/2743] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:20:10 2022] PUT /identity/v3/projects/3d2eaffc522042d981e4496050bf71ed/users/05d5004da6b14a26843a55faa346c6cd/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:20:12.514777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:12.604293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6d35c-88b2-4bca-aa6b-add9147e46db admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:12.698220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:12.699435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:12.699793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:12.700122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:12.703112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:12.759015 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:12.802692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:12.803768 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:12.817889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1367/2744] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:20:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:16.880169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7372ff-35c6-4f6e-a4e5-b979e5ef635a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:16.881904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7372ff-35c6-4f6e-a4e5-b979e5ef635a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:16.882798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7372ff-35c6-4f6e-a4e5-b979e5ef635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:16.942803 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4b7372ff-35c6-4f6e-a4e5-b979e5ef635a None None] MFA Rules not processed for user `05d5004da6b14a26843a55faa346c6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:16.944161 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4b7372ff-35c6-4f6e-a4e5-b979e5ef635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:17.104094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1378/2745] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:20:17.119103 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:17.121124 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:17.241565 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f962-8cee-4e34-8503-2d595e1a3a36 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:17.249804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:17.250491 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:17.250891 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:17.251132 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:17.324350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:17.325980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:17.339025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1368/2746] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:27.095436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:27.333448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=65 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:47.082031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:47.083471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:47.083471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:47.106495 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] MFA Rules not processed for user `3db86b8d51074eec9ea8a10cacb8a602`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:20:47.107801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:47.116169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1379/2747] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:20:47.464050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:20:47.465936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:47.479425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfd5972-17a6-455a-a170-ff2170451ca8 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:20:47.483688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:20:47.484324 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:20:47.484585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:20:47.484854 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:20:47.486667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:20:47.496457 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:20:47.497871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:20:47.506013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1369/2748] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:20:57.100893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:20:57.502297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:21:05.931829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:21:05.943496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c660ac-5b0b-42cd-bf8a-c74f0a1fb773 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:21:05.947643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c4d81-46ca-4e64-8528-134227f4ff70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:21:05.948321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c4d81-46ca-4e64-8528-134227f4ff70 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:21:05.948653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c4d81-46ca-4e64-8528-134227f4ff70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:21:05.948926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c4d81-46ca-4e64-8528-134227f4ff70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:21:05.958836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c4d81-46ca-4e64-8528-134227f4ff70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:21:05.960288 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c4d81-46ca-4e64-8528-134227f4ff70 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:21:05.969868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1380/2749] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:21:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:21:15.965698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:22:50.204882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:50.220162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca034-266c-42e0-ad39-7c30628db2ca service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:50.226002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:50.226728 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:50.227114 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:50.227461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:50.231036 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:22:50.243097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:50.245523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:50.259394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1370/2750] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:22:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:22:58.624394 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:58.624880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:58.624978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:58.642988 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] MFA Rules not processed for user `d61f6d6395f54f32adf1ba1ccdb6bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:22:58.644183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:22:58.700958 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1381/2751] 10.222.0.32 () {58 vars in 967 bytes} [Wed May 25 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:22:58.741398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:22:58.742903 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:22:58.825129 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eaade8-4c7a-401e-ab3a-776cfd5a9ba2 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:22:58.830936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5808b2b191a14c358ec62171be87a01f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:22:58.831572 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:22:58.831848 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:22:58.832118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:22:58.840477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:22:58.842044 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:22:58.850992 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1371/2752] 10.222.0.32 () {60 vars in 1384 bytes} [Wed May 25 15:22:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:23:06.003813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:06.005859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:06.074620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2dfa3-d2f4-4255-bcf2-4fcf17e2a8ac None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:06.079156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:06.080318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:06.080622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:06.080895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] PATH_INFO: `/v3/users/3f41efe7f64f41698b5f9ce002c7da71` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:06.082086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f41efe7f64f41698b5f9ce002c7da71)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:06.084065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:06.133477 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f41efe7f64f41698b5f9ce002c7da71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.134203 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f41efe7f64f41698b5f9ce002c7da71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.134856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f41efe7f64f41698b5f9ce002c7da71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.146519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f41efe7f64f41698b5f9ce002c7da71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.153642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f41efe7f64f41698b5f9ce002c7da71'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.167876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1382/2753] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:23:06 2022] DELETE /identity/v3/users/3f41efe7f64f41698b5f9ce002c7da71 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:23:06.347424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:06.358462 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e78cd8-5f88-40d8-acf3-511223d4f3ea service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:06.418888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:06.420030 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:06.420314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:06.420589 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] PATH_INFO: `/v3/projects/630246769a674d5b9c5437dfa07b3903` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:06.421804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=630246769a674d5b9c5437dfa07b3903)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:06.423770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:06.481056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630246769a674d5b9c5437dfa07b3903', 'request_id': 'req-7a853a4c-c71c-4448-98c1-cd193940b448'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.491879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630246769a674d5b9c5437dfa07b3903', 'request_id': 'req-7a853a4c-c71c-4448-98c1-cd193940b448'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.499118 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630246769a674d5b9c5437dfa07b3903', 'request_id': 'req-7a853a4c-c71c-4448-98c1-cd193940b448'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.501773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:06.515238 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] The token cache is being invalidate because project 630246769a674d5b9c5437dfa07b3903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:23:06.515601 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.516305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.517988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1372/2754] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:23:06 2022] DELETE /identity/v3/projects/630246769a674d5b9c5437dfa07b3903 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:23:06.522162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:06.523831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:06.537346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb419f3-1e15-48e0-bb1f-f9269de1be16 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:06.556931 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:06.603454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:06.604327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:06.604608 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:06.604880 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] PATH_INFO: `/v3/users/24e65a8a92e3472fb5b043361899d9e6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:06.606054 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e65a8a92e3472fb5b043361899d9e6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:06.608044 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:06.656549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e65a8a92e3472fb5b043361899d9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.657187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e65a8a92e3472fb5b043361899d9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.657791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e65a8a92e3472fb5b043361899d9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.670907 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e65a8a92e3472fb5b043361899d9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.682133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb5506e8-68e9-42ae-9839-68635f7c14d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e65a8a92e3472fb5b043361899d9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:06.699584 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1383/2755] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:23:06 2022] DELETE /identity/v3/users/24e65a8a92e3472fb5b043361899d9e6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:23:06.894136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:06.895687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:06.912145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a853a4c-c71c-4448-98c1-cd193940b448 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:06.973608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:06.974478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:06.974777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:06.975088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] PATH_INFO: `/v3/projects/81c5c7d75b764ae096ce20b8cc5bf12d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:06.976272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c5c7d75b764ae096ce20b8cc5bf12d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:06.978294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.026397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c5c7d75b764ae096ce20b8cc5bf12d', 'request_id': 'req-8fd6468b-bd75-4542-b966-c6e52757bca3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:07.032339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c5c7d75b764ae096ce20b8cc5bf12d', 'request_id': 'req-8fd6468b-bd75-4542-b966-c6e52757bca3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:07.037759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c5c7d75b764ae096ce20b8cc5bf12d', 'request_id': 'req-8fd6468b-bd75-4542-b966-c6e52757bca3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:07.051112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] The token cache is being invalidate because project 81c5c7d75b764ae096ce20b8cc5bf12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:23:07.051329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:07.052039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8fd6468b-bd75-4542-b966-c6e52757bca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:23:07.053668 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1373/2756] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:23:06 2022] DELETE /identity/v3/projects/81c5c7d75b764ae096ce20b8cc5bf12d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:23:07.058943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.059433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.059530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.078200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:07.079459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:07.140503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1384/2757] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:07.148434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.148718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.149126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.168664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:07.169882 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:07.178296 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1374/2758] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:07.182572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.192740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b945264-1b39-485c-b16c-714a9f3cee33 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.200208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.201162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.201509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.201850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.203033 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:23:07.203420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.206279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.220468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1385/2759] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:23:07.224823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.234587 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81df00-7410-490b-a91c-549f603844a7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.240628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.241275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.241562 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.241825 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.242539 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.244533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.277154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1375/2760] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:23:07 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:23:07.281613 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.293529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c42730-87a7-4dc6-9e2f-57b4cefafb60 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.328187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:07.360127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.360883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.361281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.361654 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.362690 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.365773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.387869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1386/2761] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:23:07 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:23:07.391932 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.402350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81b58-9f08-454d-a7a5-74ade6340edd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.407310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.407993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.408268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.408563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.409283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.410874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.421632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1376/2762] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:23:07.425571 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.436140 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4eafc-3aa2-4d01-a894-06380f24053f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.441183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fec6d0-d03b-4939-8203-e3dfd09dde5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.441845 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fec6d0-d03b-4939-8203-e3dfd09dde5f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.442124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fec6d0-d03b-4939-8203-e3dfd09dde5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.442397 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fec6d0-d03b-4939-8203-e3dfd09dde5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.443155 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fec6d0-d03b-4939-8203-e3dfd09dde5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.444768 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fec6d0-d03b-4939-8203-e3dfd09dde5f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.456150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1387/2763] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:23:07.460220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.470869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a5f44-47a5-4e99-8393-67c3e89dfe92 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.476038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.476933 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.477211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.477478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] PATH_INFO: `/v3/projects/9c4c13c5b10447a98c6066552f38015b/users/1311236603b34138ac25eb3d1d98929e/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.495204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4c13c5b10447a98c6066552f38015b, user_id=1311236603b34138ac25eb3d1d98929e, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.497850 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.512602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1377/2764] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:23:07 2022] PUT /identity/v3/projects/9c4c13c5b10447a98c6066552f38015b/users/1311236603b34138ac25eb3d1d98929e/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:23:07.519609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfd426-1e40-49f5-80e5-3fd66afe22df None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.519898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfd426-1e40-49f5-80e5-3fd66afe22df None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.520173 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfd426-1e40-49f5-80e5-3fd66afe22df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.538250 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8bdfd426-1e40-49f5-80e5-3fd66afe22df None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:07.539471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8bdfd426-1e40-49f5-80e5-3fd66afe22df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:07.599527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1388/2765] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:07.610535 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:07.612477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:07.635003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:07.684175 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bab817-a1bc-4973-8490-910a82d51fe5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:07.688429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:07.689007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:07.689283 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:07.689533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:07.699706 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:07.701121 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:07.713310 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1378/2766] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:23:11.485426 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.487406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.487688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.515971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] MFA Rules not processed for user `1311236603b34138ac25eb3d1d98929e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:11.517285 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:11.583661 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1389/2767] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:11.588527 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:11.598856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c0a43-6806-40a6-b341-99e08d3fe906 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:11.604163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:11.605139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.605665 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.606133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.606959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:11.609112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:11.639153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1379/2768] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:23:11 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:23:11.642651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:11.653966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c195e-a106-4c52-a80a-dda6ef4684ee None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:11.692916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:11.712226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:11.712976 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.713258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.713531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.714221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:11.716222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:11.738983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1390/2769] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:23:11 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:23:11.743064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:11.755368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a6c3-1999-471f-8882-3bc73abbd886 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:11.760875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:11.761564 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.761841 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.762105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.762860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:11.764529 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:11.776199 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1380/2770] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:23:11.780796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:11.791795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d068e2-b4b6-4bff-bf8d-d84002b44e68 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:11.796794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:11.797829 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.798171 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.798507 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] PATH_INFO: `/v3/projects/89e825a0ce8e4bb583196114cf33bb53/users/5441bc5121484ac5a706705e8fa34cfe/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.815811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e825a0ce8e4bb583196114cf33bb53, user_id=5441bc5121484ac5a706705e8fa34cfe, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:11.818590 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:11.834073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1391/2771] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:23:11 2022] PUT /identity/v3/projects/89e825a0ce8e4bb583196114cf33bb53/users/5441bc5121484ac5a706705e8fa34cfe/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:23:11.838227 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:11.848559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4b915-5fb4-4313-b608-9eb6f151d671 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:11.904653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6304f11-52a6-4da2-a722-487eca9d99bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:11.905320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6304f11-52a6-4da2-a722-487eca9d99bc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.905600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6304f11-52a6-4da2-a722-487eca9d99bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.905869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6304f11-52a6-4da2-a722-487eca9d99bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.906592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6304f11-52a6-4da2-a722-487eca9d99bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:11.908222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6304f11-52a6-4da2-a722-487eca9d99bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:11.918501 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1381/2772] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:23:11.922629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:11.933710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad187d-c947-4d79-a5ba-4a34d57c44a2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:11.938815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:11.939725 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:11.940005 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:11.940405 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] PATH_INFO: `/v3/projects/89e825a0ce8e4bb583196114cf33bb53/users/5441bc5121484ac5a706705e8fa34cfe/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:11.942692 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e825a0ce8e4bb583196114cf33bb53, user_id=5441bc5121484ac5a706705e8fa34cfe, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:11.945581 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:11.958060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1392/2773] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:23:11 2022] PUT /identity/v3/projects/89e825a0ce8e4bb583196114cf33bb53/users/5441bc5121484ac5a706705e8fa34cfe/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:23:16.515473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e5534-ecfc-4dba-9930-e8a2f5510b7a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:16.516959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e5534-ecfc-4dba-9930-e8a2f5510b7a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:16.517576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e5534-ecfc-4dba-9930-e8a2f5510b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:16.546341 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e8e5534-ecfc-4dba-9930-e8a2f5510b7a None None] MFA Rules not processed for user `5441bc5121484ac5a706705e8fa34cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:16.547608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e8e5534-ecfc-4dba-9930-e8a2f5510b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:16.637000 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1382/2774] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:16.652583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:16.654991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:16.740430 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406e459-e7a0-4b13-9b44-aa856db89f35 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:16.746836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b63b4-738e-4859-acc0-9be34e315e84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:16.747948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b63b4-738e-4859-acc0-9be34e315e84 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:16.748235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b63b4-738e-4859-acc0-9be34e315e84 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:16.748565 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b63b4-738e-4859-acc0-9be34e315e84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:16.811097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b63b4-738e-4859-acc0-9be34e315e84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:16.812720 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b63b4-738e-4859-acc0-9be34e315e84 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:16.822711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1393/2775] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:23:17.762574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:17.763131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:17.763209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:17.781540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] MFA Rules not processed for user `1311236603b34138ac25eb3d1d98929e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:17.782808 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:17.789935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1383/2776] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:26.818310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:27.779069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:28.942222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:28.942700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:28.942818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:28.962427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] MFA Rules not processed for user `3db86b8d51074eec9ea8a10cacb8a602`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:23:28.963669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:29.023622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1394/2777] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:23:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:23:29.336419 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:29.392727 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2e3df-a9d4-45b5-9405-bed1f0c7bbda None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:29.396678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:29.397331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:29.397632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:29.397904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:29.399720 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:29.409379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:29.410860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:29.418940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1384/2778] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:23:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:23:39.016494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:39.415483 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:23:48.594138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:23:48.624000 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c0b74-30d5-4b3d-bbbf-91551f9304be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:23:48.635127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:23:48.635985 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:23:48.636268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:23:48.636548 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:23:48.638423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:23:48.656208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:23:48.657833 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:23:48.671983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1395/2779] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:23:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:23:58.666208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:04.335003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:04.344964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118f06-49c1-469b-b43f-feb05208a397 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:04.349500 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:04.350137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:04.350413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:04.350692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:04.360644 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:04.362083 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:04.371956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1385/2780] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:14.367823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:22.869964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:22.871796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:22.980689 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567faa0-26f2-4e57-95bf-37cbbda2a127 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:22.984627 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:22.985275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:22.985552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:22.985848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:22.987668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:23.061299 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:23.062589 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:23.072021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1396/2781] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:23.207428 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:23.216511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22e417-4c9a-4456-bfb2-4fb648e59078 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:23.220110 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:23.220736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:23.221012 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:23.221281 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:23.223134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:23.290894 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:23.293709 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:23.311621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1386/2782] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:26.458529 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:26.460192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:26.532633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:26.535977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:26.556369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:26.560216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d42f1-253d-489e-8e97-05983b690bb8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:26.565726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:26.566807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:26.567095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:26.567372 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] PATH_INFO: `/v3/users/2401371a775743c29c16406cb9df5d28` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:26.568576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2401371a775743c29c16406cb9df5d28)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:26.570525 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:26.573474 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0ac7e-9b48-4feb-8305-ef952f770f23 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:26.578603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:26.580006 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:26.580287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:26.580569 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:26.582544 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:26.594822 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:26.596604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:26.607707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1397/2783] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:26.643654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2401371a775743c29c16406cb9df5d28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.644305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2401371a775743c29c16406cb9df5d28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.651285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2401371a775743c29c16406cb9df5d28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.652003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2401371a775743c29c16406cb9df5d28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.663858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-38194db9-855e-4bb5-b207-a24a99404bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2401371a775743c29c16406cb9df5d28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.680925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1387/2784] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:26 2022] DELETE /identity/v3/users/2401371a775743c29c16406cb9df5d28 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:26.852151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:26.864465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20be4a3-5ca8-4f92-a384-52bcbaf8a13a service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:26.923205 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:26.924075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:26.924356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:26.924625 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] PATH_INFO: `/v3/projects/16613c1e95ff4565a9ed9393295064a2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:26.925821 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16613c1e95ff4565a9ed9393295064a2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:26.927800 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:26.978031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16613c1e95ff4565a9ed9393295064a2', 'request_id': 'req-d09c56d5-c1ce-4402-9015-771c15846d9f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.984101 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16613c1e95ff4565a9ed9393295064a2', 'request_id': 'req-d09c56d5-c1ce-4402-9015-771c15846d9f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:26.989826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16613c1e95ff4565a9ed9393295064a2', 'request_id': 'req-d09c56d5-c1ce-4402-9015-771c15846d9f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:27.003573 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] The token cache is being invalidate because project 16613c1e95ff4565a9ed9393295064a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:27.003864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:27.004606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d09c56d5-c1ce-4402-9015-771c15846d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:27.006199 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1398/2785] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:26 2022] DELETE /identity/v3/projects/16613c1e95ff4565a9ed9393295064a2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:27.012967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.013256 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.013541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.032094 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:27.033315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:27.087877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:27.101987 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1388/2786] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:27.112848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.113129 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.113398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.133352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:27.134574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:27.142745 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1399/2787] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:27.147208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.157172 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999f155-0287-4df5-9db8-d3e488342a79 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.162113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.162777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.163065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.163332 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.164056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:24:27.164359 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.165975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.178086 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1389/2788] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:24:27.182570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.192467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932ae7a-7008-443a-bc17-c808cc0851a6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.198233 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.198948 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.199227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.199494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.200183 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.202102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.230497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1400/2789] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:27 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:27.235115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.245459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c459c-6843-4f3a-afac-47f5e6076adc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.303923 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.304584 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.304856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.305116 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.305778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.307725 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.336467 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1390/2790] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:27 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:27.340564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.351075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f96f2c-8c80-4723-9dfa-439c50a83683 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.355971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.356707 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.357025 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.357331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.358172 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.359871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.370467 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1401/2791] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:27.374533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.384900 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecdb66-a2f1-45cc-b7de-495ec70ad477 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.389701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0431380c-6ae4-4545-976f-06ca8a35f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.390383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431380c-6ae4-4545-976f-06ca8a35f4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.390661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431380c-6ae4-4545-976f-06ca8a35f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.390960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0431380c-6ae4-4545-976f-06ca8a35f4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.391682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431380c-6ae4-4545-976f-06ca8a35f4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.393276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0431380c-6ae4-4545-976f-06ca8a35f4a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.403692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1391/2792] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:27.408393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.418561 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaebf53-5c28-4360-bbdc-7fc96227ebb0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.423854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.424733 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.425010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.425276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] PATH_INFO: `/v3/projects/940be5b5edd64bd2bee12adf5628bdb2/users/19cc933654334f518bcb4eda304b9c87/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.442065 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] RBAC: Authorizing `identity:create_grant(project_id=940be5b5edd64bd2bee12adf5628bdb2, user_id=19cc933654334f518bcb4eda304b9c87, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.444784 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.458732 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1402/2793] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:27 2022] PUT /identity/v3/projects/940be5b5edd64bd2bee12adf5628bdb2/users/19cc933654334f518bcb4eda304b9c87/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:27.469870 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.470185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.470454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.498191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] MFA Rules not processed for user `19cc933654334f518bcb4eda304b9c87`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:27.499334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:27.565035 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1392/2794] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:27.576275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.578204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:27.589513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:27.655396 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c56d2-25a4-4f59-8f08-df4ba56ad83f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.661321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46285634-06de-4922-99e9-7e1a586cc5be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.661970 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46285634-06de-4922-99e9-7e1a586cc5be service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.662257 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46285634-06de-4922-99e9-7e1a586cc5be service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.662532 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46285634-06de-4922-99e9-7e1a586cc5be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.670344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46285634-06de-4922-99e9-7e1a586cc5be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.671979 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46285634-06de-4922-99e9-7e1a586cc5be service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:27.680754 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1403/2795] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:27.905435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:27.966946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:27.972964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f219ae-fec5-434b-847d-30008ade88b4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:27.979140 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:27.980018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:27.980297 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:27.981072 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] PATH_INFO: `/v3/users/19cc933654334f518bcb4eda304b9c87` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:27.981806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19cc933654334f518bcb4eda304b9c87)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:27.983814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:28.033544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19cc933654334f518bcb4eda304b9c87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.034205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19cc933654334f518bcb4eda304b9c87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.041090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19cc933654334f518bcb4eda304b9c87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.041718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19cc933654334f518bcb4eda304b9c87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.053486 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19cc933654334f518bcb4eda304b9c87'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.066940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1393/2796] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:27 2022] DELETE /identity/v3/users/19cc933654334f518bcb4eda304b9c87 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:28.071614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.071918 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.072190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.090723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:28.091943 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:28.154273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1404/2797] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:24:28.165790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:28.167435 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:28.226060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621165b-e099-4f54-8249-1490b81c5c31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:28.230054 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbb957-b3f6-4610-b97d-a8f4d6db4873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:28.230701 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbb957-b3f6-4610-b97d-a8f4d6db4873 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.231018 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbb957-b3f6-4610-b97d-a8f4d6db4873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.231292 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbb957-b3f6-4610-b97d-a8f4d6db4873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.240890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbb957-b3f6-4610-b97d-a8f4d6db4873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:28.242313 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbb957-b3f6-4610-b97d-a8f4d6db4873 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:28.252305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1394/2798] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:28.527318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:28.528965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:28.543864 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b153-b925-4421-94d9-d50b576d3887 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:28.548752 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:28.549629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.549910 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.550196 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] PATH_INFO: `/v3/projects/940be5b5edd64bd2bee12adf5628bdb2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.551435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940be5b5edd64bd2bee12adf5628bdb2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:28.554182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:28.605068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940be5b5edd64bd2bee12adf5628bdb2', 'request_id': 'req-337990c1-6d81-4c1c-8882-3241a5c11f58'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.611156 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940be5b5edd64bd2bee12adf5628bdb2', 'request_id': 'req-337990c1-6d81-4c1c-8882-3241a5c11f58'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.616959 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940be5b5edd64bd2bee12adf5628bdb2', 'request_id': 'req-337990c1-6d81-4c1c-8882-3241a5c11f58'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.630550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] The token cache is being invalidate because project 940be5b5edd64bd2bee12adf5628bdb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:28.630872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.631678 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-337990c1-6d81-4c1c-8882-3241a5c11f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:28.633482 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1405/2799] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:28 2022] DELETE /identity/v3/projects/940be5b5edd64bd2bee12adf5628bdb2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:28.638977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.639583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.639583 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.657473 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:28.658678 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:28.719046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1395/2800] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:24:28.728483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.728868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.729232 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.748884 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:28.750093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:28.758631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1406/2801] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:28.763257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:28.773440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a33b5d-c58b-41bc-8515-4bb71dd48e2a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:28.779095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:28.779759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.780038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.780311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.781041 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:24:28.781345 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:28.782969 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:28.795907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1396/2802] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:24:28.801454 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:28.816184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8d2df-8d55-4850-9994-624503a9bcf0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:28.823305 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:28.824284 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.824723 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.825154 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.826258 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:28.829515 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:28.835785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:28.864136 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1407/2803] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:28 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:28.868418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:28.875009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:28.880139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1b551-57dd-4186-9c7e-cb2156305523 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:28.944204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:28.944905 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.945162 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.945431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.946130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:28.948091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:28.968716 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1397/2804] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:28 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:28.973224 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:28.984188 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7aeba-3cf0-4ab0-8600-4340418ad21e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:28.989290 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:28.989965 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:28.990244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:28.990556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:28.991303 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:28.992935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.003331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1408/2805] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:29.007445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.017858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ce910-a48d-4162-b352-2409724bdfc5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.023205 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400da40-5fc4-4331-896e-58380359f76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.023887 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400da40-5fc4-4331-896e-58380359f76d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.024164 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400da40-5fc4-4331-896e-58380359f76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.024431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400da40-5fc4-4331-896e-58380359f76d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.025142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400da40-5fc4-4331-896e-58380359f76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.026742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400da40-5fc4-4331-896e-58380359f76d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.038396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1398/2806] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:29.043087 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.053342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a91e29-8ffe-44fe-bc9d-85f473e2ff12 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.058084 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.058993 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.059270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.059539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] PATH_INFO: `/v3/projects/4094914b08a64fb28a76e8b9f30256b4/users/1a42a67b4362473d87071c6296644b6a/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.074668 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4094914b08a64fb28a76e8b9f30256b4, user_id=1a42a67b4362473d87071c6296644b6a, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.077384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.091932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1409/2807] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:29 2022] PUT /identity/v3/projects/4094914b08a64fb28a76e8b9f30256b4/users/1a42a67b4362473d87071c6296644b6a/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:29.102373 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.102656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.102956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.130633 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] MFA Rules not processed for user `1a42a67b4362473d87071c6296644b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:29.131856 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:29.203660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1399/2808] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:29.208314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.273854 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e6b90-332b-4c4b-9d09-199f0fcfd8fa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.278933 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.279596 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.279895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.280139 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.280831 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.282866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.314682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1410/2809] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:29 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:29.319354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.330436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b474c5-4029-47af-a57e-c39979b9f447 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.394400 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.395034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.395312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.395577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.396269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.398178 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.413790 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:29.421750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1400/2810] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:29 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:29.425836 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.436786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204a01-c543-48d0-9aaf-f3af81c2b23a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.441940 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.442609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.442917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.443186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.443904 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.445504 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.457393 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1411/2811] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:29.461975 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.472776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119884be-9ba3-431e-a17f-265ee38a0b62 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.478217 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.479142 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.479423 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.479697 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] PATH_INFO: `/v3/projects/dc1827843598474d97a1d42ca471c036/users/96f34d834980428786a947f82a8bef5a/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.496156 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1827843598474d97a1d42ca471c036, user_id=96f34d834980428786a947f82a8bef5a, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.498849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.514184 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1401/2812] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:29 2022] PUT /identity/v3/projects/dc1827843598474d97a1d42ca471c036/users/96f34d834980428786a947f82a8bef5a/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:29.518651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.529037 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df950039-8752-4357-ab9a-f51f0da92cf6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.591849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72764589-063e-45e3-9dfe-d49d207bc986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.592531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764589-063e-45e3-9dfe-d49d207bc986 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.592801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764589-063e-45e3-9dfe-d49d207bc986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.593078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72764589-063e-45e3-9dfe-d49d207bc986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.593801 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72764589-063e-45e3-9dfe-d49d207bc986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.595449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72764589-063e-45e3-9dfe-d49d207bc986 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.606122 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1412/2813] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:29.610829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.621950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf245-5874-4d53-9828-53304679d014 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.627600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.628506 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.628791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.629064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] PATH_INFO: `/v3/projects/dc1827843598474d97a1d42ca471c036/users/96f34d834980428786a947f82a8bef5a/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.631314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1827843598474d97a1d42ca471c036, user_id=96f34d834980428786a947f82a8bef5a, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.633972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.645433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1402/2814] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:29 2022] PUT /identity/v3/projects/dc1827843598474d97a1d42ca471c036/users/96f34d834980428786a947f82a8bef5a/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:29.655789 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.656121 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.656434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.684473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] MFA Rules not processed for user `96f34d834980428786a947f82a8bef5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:29.685708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:29.690997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:29.756865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1413/2815] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:29.768523 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.770388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:29.837554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99960bc-c491-4f04-8b8e-5e5586c5874b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.841468 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682695db-6302-4a92-be27-b62a8809b568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.842100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682695db-6302-4a92-be27-b62a8809b568 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.842386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682695db-6302-4a92-be27-b62a8809b568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.842654 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682695db-6302-4a92-be27-b62a8809b568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.851972 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682695db-6302-4a92-be27-b62a8809b568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.853394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682695db-6302-4a92-be27-b62a8809b568 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:29.862940 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1403/2816] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:29.891653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:29.956483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c4f1f-baee-448b-b7cf-3f3317a1f727 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:29.962634 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:29.963574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:29.963855 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:29.964127 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] PATH_INFO: `/v3/users/1a42a67b4362473d87071c6296644b6a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:29.965449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a42a67b4362473d87071c6296644b6a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:29.967582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:30.018526 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a42a67b4362473d87071c6296644b6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.019204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a42a67b4362473d87071c6296644b6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.019811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a42a67b4362473d87071c6296644b6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.032848 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a42a67b4362473d87071c6296644b6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.040497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a42a67b4362473d87071c6296644b6a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.055519 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1414/2817] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:29 2022] DELETE /identity/v3/users/1a42a67b4362473d87071c6296644b6a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:30.061237 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:30.061542 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:30.061811 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:30.080667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:30.081872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:30.142973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1404/2818] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:24:30.153438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:30.154973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:30.183769 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:30.219324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea375139-4da0-4356-a597-b0d9ccef78e8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:30.223268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:30.223926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:30.224206 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:30.224486 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:30.233975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:30.235506 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:30.244955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1415/2819] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:30.520946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:30.523154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:30.541125 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363121f-08cd-4b32-99b6-9b89438ebd71 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:30.547299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:30.548176 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:30.548451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:30.548714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] PATH_INFO: `/v3/projects/4094914b08a64fb28a76e8b9f30256b4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:30.549918 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4094914b08a64fb28a76e8b9f30256b4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:30.551973 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:30.602976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4094914b08a64fb28a76e8b9f30256b4', 'request_id': 'req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.604931 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:30.609876 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4094914b08a64fb28a76e8b9f30256b4', 'request_id': 'req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.616961 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4094914b08a64fb28a76e8b9f30256b4', 'request_id': 'req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.630049 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] The token cache is being invalidate because project 4094914b08a64fb28a76e8b9f30256b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:30.630336 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.631010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.632679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1405/2820] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:30 2022] DELETE /identity/v3/projects/4094914b08a64fb28a76e8b9f30256b4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:30.636722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:30.646456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f992a-3eff-4152-b318-defe3fe52b1f service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:30.701774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:30.702659 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:30.702966 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:30.703236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] PATH_INFO: `/v3/users/96f34d834980428786a947f82a8bef5a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:30.704498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f34d834980428786a947f82a8bef5a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:30.706461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:30.755078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f34d834980428786a947f82a8bef5a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.755805 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f34d834980428786a947f82a8bef5a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.756514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f34d834980428786a947f82a8bef5a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.769595 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f34d834980428786a947f82a8bef5a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.777068 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d723610b-d9ab-45e9-8a5c-b350771121a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f34d834980428786a947f82a8bef5a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:30.791658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1416/2821] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:30 2022] DELETE /identity/v3/users/96f34d834980428786a947f82a8bef5a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:31.071659 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.073959 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:31.091628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1330e-93c0-43b0-8fe4-f99a0b1afc93 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.150974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.151845 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.152126 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.152401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] PATH_INFO: `/v3/projects/dc1827843598474d97a1d42ca471c036` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.153666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1827843598474d97a1d42ca471c036)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.155614 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.202513 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1827843598474d97a1d42ca471c036', 'request_id': 'req-1345d679-079d-407f-bca5-3b13b5a9edb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:31.208869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1827843598474d97a1d42ca471c036', 'request_id': 'req-1345d679-079d-407f-bca5-3b13b5a9edb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:31.215454 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1827843598474d97a1d42ca471c036', 'request_id': 'req-1345d679-079d-407f-bca5-3b13b5a9edb0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:31.228747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] The token cache is being invalidate because project dc1827843598474d97a1d42ca471c036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:31.229034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:31.229671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1345d679-079d-407f-bca5-3b13b5a9edb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:31.231282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1406/2822] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:31 2022] DELETE /identity/v3/projects/dc1827843598474d97a1d42ca471c036 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:31.236574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.236900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.237212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.255924 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:31.257142 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:31.298849 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:31.321930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1417/2823] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:31.381320 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.381596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.382105 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.401955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:31.403208 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:31.412691 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1407/2824] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:24:31.417176 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.427313 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8231b0-54ea-4ea6-9f59-d7166c879ccc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.432279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.432916 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.433192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.433458 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.434186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:24:31.434487 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.436138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.448027 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1418/2825] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:24:31.452518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.462010 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8b211-4f68-4efa-8e7c-76d6ce80f8f3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.466783 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.467487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.467792 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.468061 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.468777 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.470650 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.497231 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1408/2826] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:31.501900 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.512428 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598977e4-e23c-4d2e-bb64-166e3b8466f3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.570060 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.570741 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.571046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.571318 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.572056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.573945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.596344 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1419/2827] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:31.601220 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.612262 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4491b-c1b9-4741-869c-b422af21c10b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.617761 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.618461 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.618767 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.619042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.619791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.621429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.633408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1409/2828] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:31.638093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.648895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd3509-010c-4bb5-82c9-766779d4c09d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.653698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7483890-0edf-413a-8007-bb8c8138c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.654382 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7483890-0edf-413a-8007-bb8c8138c03d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.654660 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7483890-0edf-413a-8007-bb8c8138c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.654971 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7483890-0edf-413a-8007-bb8c8138c03d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.655722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7483890-0edf-413a-8007-bb8c8138c03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.657375 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7483890-0edf-413a-8007-bb8c8138c03d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.668020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1420/2829] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:31.673831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.684350 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9d781-1b15-454f-a293-ceec3e6b7b8c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.689399 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.690314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.690585 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.690906 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] PATH_INFO: `/v3/projects/c3a2ccf3cf4d486ba3383d683e5381a3/users/a0f8a06bcacc4fb2993eaf92b05aa1ad/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.706658 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a2ccf3cf4d486ba3383d683e5381a3, user_id=a0f8a06bcacc4fb2993eaf92b05aa1ad, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.709531 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.724635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1410/2830] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:31 2022] PUT /identity/v3/projects/c3a2ccf3cf4d486ba3383d683e5381a3/users/a0f8a06bcacc4fb2993eaf92b05aa1ad/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:31.738423 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.738740 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.739147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.769711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] MFA Rules not processed for user `a0f8a06bcacc4fb2993eaf92b05aa1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:31.771126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:31.840442 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1421/2831] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:31.845853 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.858947 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:31.919211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e35e1-d910-47b5-aa4a-c1a3252c553b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:31.924271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:31.924963 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:31.925243 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:31.925528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:31.926254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:31.928284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:31.957436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1411/2832] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:31.963585 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:31.976530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891235a7-d4d8-4db1-aab7-a301a0d76c8c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.019264 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:32.040513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.041229 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.041511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.041799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.042525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.044614 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.064952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1422/2833] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:31 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:32.069443 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.079956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3411d8-db03-4407-9664-fbad9eb945ed admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.085433 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.086434 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.086738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.087042 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.087738 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.089361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.101702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1412/2834] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:32.106011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.117149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d5d81-3741-42f9-a0c0-89793934164d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.122359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b1892-7525-4461-a7b5-f2190e90d0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.123076 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b1892-7525-4461-a7b5-f2190e90d0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.123471 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b1892-7525-4461-a7b5-f2190e90d0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.123753 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b1892-7525-4461-a7b5-f2190e90d0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.124488 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b1892-7525-4461-a7b5-f2190e90d0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.126116 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b1892-7525-4461-a7b5-f2190e90d0d5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.136957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1423/2835] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:32.142714 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.154885 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae8a2e-3bf5-4054-a03e-12f67eedcabd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.160802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.161750 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.162031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.162301 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] PATH_INFO: `/v3/projects/eaddc74415ce4db285755be9258ace5f/users/9eb318da8c294ebb9d284b075f039247/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.178559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaddc74415ce4db285755be9258ace5f, user_id=9eb318da8c294ebb9d284b075f039247, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.181441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.196416 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1413/2836] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:32 2022] PUT /identity/v3/projects/eaddc74415ce4db285755be9258ace5f/users/9eb318da8c294ebb9d284b075f039247/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:32.207148 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.207434 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.207674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.235174 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] MFA Rules not processed for user `9eb318da8c294ebb9d284b075f039247`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:32.236390 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:32.301179 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1424/2837] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:32.306002 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.369021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d16fef-2155-4758-b464-4db215e0041c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.374117 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.374776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.375090 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.375366 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.376046 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.377953 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.403788 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1414/2838] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:32 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:32.408216 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.419543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca13b4-1bca-4557-ba90-8f116da3a434 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.477794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.478476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.478777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.479062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.479734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.481670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.503946 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1425/2839] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:32 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:32.508445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.519442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ceb87-becf-403c-aef4-043b51819d48 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.524802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.525460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.525737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.526039 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.526793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.528464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.539344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1415/2840] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:24:32.544182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.554633 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c18876-7873-429d-abe3-d003909db38e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.560228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.561357 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.561912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.562356 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] PATH_INFO: `/v3/projects/49e1ea0c53fa438f8cba2c4b89515591/users/f4fcd510772d4df09c4e354688a97c1e/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.578937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e1ea0c53fa438f8cba2c4b89515591, user_id=f4fcd510772d4df09c4e354688a97c1e, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.581688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.597059 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1426/2841] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:32 2022] PUT /identity/v3/projects/49e1ea0c53fa438f8cba2c4b89515591/users/f4fcd510772d4df09c4e354688a97c1e/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:32.601284 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.611590 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eb9cb-7b05-4af6-ac6b-80b1dc14daf2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.641828 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:32.672133 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee82552-c236-4431-a207-a0c49ca7a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.672802 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82552-c236-4431-a207-a0c49ca7a106 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.673080 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82552-c236-4431-a207-a0c49ca7a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.673351 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82552-c236-4431-a207-a0c49ca7a106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.674065 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee82552-c236-4431-a207-a0c49ca7a106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.675708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee82552-c236-4431-a207-a0c49ca7a106 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.685842 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1416/2842] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:32.690511 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.701631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61806d37-832d-4337-acc3-85c91e2da2aa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.707431 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.708404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.708684 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.708954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] PATH_INFO: `/v3/projects/49e1ea0c53fa438f8cba2c4b89515591/users/f4fcd510772d4df09c4e354688a97c1e/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.711528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e1ea0c53fa438f8cba2c4b89515591, user_id=f4fcd510772d4df09c4e354688a97c1e, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.714236 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.721202 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:32.726103 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1427/2843] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:32 2022] PUT /identity/v3/projects/49e1ea0c53fa438f8cba2c4b89515591/users/f4fcd510772d4df09c4e354688a97c1e/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:32.738453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.738745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.739084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.765724 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] MFA Rules not processed for user `f4fcd510772d4df09c4e354688a97c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:32.766984 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:32.840573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1417/2844] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:32.852632 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.854700 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:32.914126 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72106b1c-be38-46fa-b765-c702d7ee127f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:32.917926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:32.918570 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:32.918878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:32.919146 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:32.928528 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:32.929995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:32.939352 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1428/2845] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:32.983364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:32.985024 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:32.998050 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa9c68-68ac-41b2-a391-cb7053921c43 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:33.002413 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fb80c-7e14-435c-a45f-395f5f63ae32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:33.003100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fb80c-7e14-435c-a45f-395f5f63ae32 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:33.003385 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fb80c-7e14-435c-a45f-395f5f63ae32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:33.003660 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fb80c-7e14-435c-a45f-395f5f63ae32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:33.062544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fb80c-7e14-435c-a45f-395f5f63ae32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:33.064025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fb80c-7e14-435c-a45f-395f5f63ae32 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:33.073259 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1418/2846] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:35.997794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:36.064963 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73de95d-e061-4b7c-a67a-1940dcccdccb service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:36.070370 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:36.071295 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:36.071572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:36.071840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] PATH_INFO: `/v3/users/a0f8a06bcacc4fb2993eaf92b05aa1ad` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:36.073214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f8a06bcacc4fb2993eaf92b05aa1ad)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:36.076220 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:36.144903 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f8a06bcacc4fb2993eaf92b05aa1ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.146766 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f8a06bcacc4fb2993eaf92b05aa1ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.147520 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f8a06bcacc4fb2993eaf92b05aa1ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.163075 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f8a06bcacc4fb2993eaf92b05aa1ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.170483 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f8a06bcacc4fb2993eaf92b05aa1ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.186064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1429/2847] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:35 2022] DELETE /identity/v3/users/a0f8a06bcacc4fb2993eaf92b05aa1ad => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:36.191489 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:36.191980 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:36.192130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:36.214520 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:36.215988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:36.264233 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:36.284800 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1419/2848] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:36.294954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:36.296516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:36.326844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:36.361366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef2a5-99b6-4d5a-909d-2fbf2cdb898e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:36.365325 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:36.365968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:36.366242 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:36.366527 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:36.375353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:36.376798 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:36.386289 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1430/2849] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:24:36.661130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:36.662577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:36.679354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224b03d-e388-4f1d-93f8-ccc09f34d0a7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:36.684422 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:36.685302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:36.685586 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:36.685927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] PATH_INFO: `/v3/projects/c3a2ccf3cf4d486ba3383d683e5381a3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:36.687187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a2ccf3cf4d486ba3383d683e5381a3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:36.689145 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:36.744357 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a2ccf3cf4d486ba3383d683e5381a3', 'request_id': 'req-11f7621e-ca21-42e0-a1a0-bcce343df3d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.750956 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a2ccf3cf4d486ba3383d683e5381a3', 'request_id': 'req-11f7621e-ca21-42e0-a1a0-bcce343df3d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.756879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a2ccf3cf4d486ba3383d683e5381a3', 'request_id': 'req-11f7621e-ca21-42e0-a1a0-bcce343df3d8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.770352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] The token cache is being invalidate because project c3a2ccf3cf4d486ba3383d683e5381a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:36.770641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.771322 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.772943 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1420/2850] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:36 2022] DELETE /identity/v3/projects/c3a2ccf3cf4d486ba3383d683e5381a3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:36.777181 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:36.787583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afefa1d-ea11-4d53-be66-8efeaafa61ba service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:36.849536 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:36.850425 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:36.850703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:36.851030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] PATH_INFO: `/v3/users/9eb318da8c294ebb9d284b075f039247` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:36.852211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb318da8c294ebb9d284b075f039247)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:36.854227 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:36.914497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb318da8c294ebb9d284b075f039247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.915266 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb318da8c294ebb9d284b075f039247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.915989 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb318da8c294ebb9d284b075f039247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.929860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb318da8c294ebb9d284b075f039247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.937695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb318da8c294ebb9d284b075f039247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:36.953510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1431/2851] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:36 2022] DELETE /identity/v3/users/9eb318da8c294ebb9d284b075f039247 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:37.251303 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:37.252793 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:37.267902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7621e-ca21-42e0-a1a0-bcce343df3d8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:37.331886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:37.332744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:37.333023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:37.333273 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] PATH_INFO: `/v3/projects/eaddc74415ce4db285755be9258ace5f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:37.334994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaddc74415ce4db285755be9258ace5f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:37.337263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:37.387071 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaddc74415ce4db285755be9258ace5f', 'request_id': 'req-598f3eb4-f30c-427d-a486-fedca34b2714'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.393216 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaddc74415ce4db285755be9258ace5f', 'request_id': 'req-598f3eb4-f30c-427d-a486-fedca34b2714'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.398901 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaddc74415ce4db285755be9258ace5f', 'request_id': 'req-598f3eb4-f30c-427d-a486-fedca34b2714'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.411927 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] The token cache is being invalidate because project eaddc74415ce4db285755be9258ace5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:37.412226 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.412979 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.414648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1421/2852] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:37 2022] DELETE /identity/v3/projects/eaddc74415ce4db285755be9258ace5f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:37.418779 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:37.420331 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:37.436208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9887362-e6ae-44f3-9678-91f037c4a5bf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:37.493418 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:37.510377 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:37.511307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:37.511588 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:37.511859 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] PATH_INFO: `/v3/users/f4fcd510772d4df09c4e354688a97c1e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:37.513067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4fcd510772d4df09c4e354688a97c1e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:37.515031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:37.567686 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4fcd510772d4df09c4e354688a97c1e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.568334 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4fcd510772d4df09c4e354688a97c1e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.568987 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4fcd510772d4df09c4e354688a97c1e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.580969 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4fcd510772d4df09c4e354688a97c1e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.587960 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8d446b7d-f302-4c19-b87b-e8861089880b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4fcd510772d4df09c4e354688a97c1e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:37.602914 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1432/2853] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:37 2022] DELETE /identity/v3/users/f4fcd510772d4df09c4e354688a97c1e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:37.878684 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:37.880150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:37.890630 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:37.895269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3eb4-f30c-427d-a486-fedca34b2714 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:37.958183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:37.958999 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:37.959279 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:37.959549 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] PATH_INFO: `/v3/projects/49e1ea0c53fa438f8cba2c4b89515591` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:37.960722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e1ea0c53fa438f8cba2c4b89515591)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:37.963009 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.015429 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e1ea0c53fa438f8cba2c4b89515591', 'request_id': 'req-f5455950-2e7c-45be-8fff-7ad899d8d891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:38.022403 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e1ea0c53fa438f8cba2c4b89515591', 'request_id': 'req-f5455950-2e7c-45be-8fff-7ad899d8d891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:38.029197 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e1ea0c53fa438f8cba2c4b89515591', 'request_id': 'req-f5455950-2e7c-45be-8fff-7ad899d8d891'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:38.044163 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] The token cache is being invalidate because project 49e1ea0c53fa438f8cba2c4b89515591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:38.044453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:38.045097 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5455950-2e7c-45be-8fff-7ad899d8d891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:38.046695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1422/2854] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:37 2022] DELETE /identity/v3/projects/49e1ea0c53fa438f8cba2c4b89515591 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:38.053062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.053347 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.053621 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.071886 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:38.073082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:38.133699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1433/2855] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:38.142177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.142464 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.142736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.162394 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:38.163649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:38.172801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1423/2856] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:38.177207 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.187576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580427b-0fb5-4380-93f1-b3caca48fb64 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.193328 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.193978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.194261 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.194534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.195304 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:24:38.195606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.197214 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.209459 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1434/2857] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:24:38.214087 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.223442 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59775a8-9e2a-4137-8e18-153b9ed7573a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.228386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.229028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.229311 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.229579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.230263 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.232211 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.261868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1424/2858] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:38.266307 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.276438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562f72b-943f-45e9-a3e9-9c95811cd5f2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.325404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:38.338276 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.338995 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.339277 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.339554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.340244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.342190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.361906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1435/2859] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:38.366043 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.377158 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb139a-f79b-4354-aeec-12fba7b7114d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.383209 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.383872 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.384150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.384418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.385131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.386717 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.398276 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1425/2860] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:38.402455 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.413192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3980e-1894-4f1c-b2a3-4d4163867622 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.418456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4a0ec-f4ea-47bf-a5c9-edc258b314d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.419241 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4a0ec-f4ea-47bf-a5c9-edc258b314d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.419812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4a0ec-f4ea-47bf-a5c9-edc258b314d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.419812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4a0ec-f4ea-47bf-a5c9-edc258b314d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.420502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4a0ec-f4ea-47bf-a5c9-edc258b314d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.422117 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4a0ec-f4ea-47bf-a5c9-edc258b314d2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.432598 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1436/2861] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:38.436671 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.447166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61b871-89ac-4c7c-a0a9-eafe1eb426d4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.452747 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.453646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.453924 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.454194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] PATH_INFO: `/v3/projects/dbd0ae6599d64380b9646f488fa437f5/users/ecf42a27073243fe94fd739ceb5a72b5/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.469411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd0ae6599d64380b9646f488fa437f5, user_id=ecf42a27073243fe94fd739ceb5a72b5, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.472048 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.485764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1426/2862] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:38 2022] PUT /identity/v3/projects/dbd0ae6599d64380b9646f488fa437f5/users/ecf42a27073243fe94fd739ceb5a72b5/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:38.498061 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.498351 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.498630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.526294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] MFA Rules not processed for user `ecf42a27073243fe94fd739ceb5a72b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:38.527552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:38.597332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1437/2863] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:38.601935 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.639574 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:38.671639 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c1d00-cd10-4d23-9913-d972aa2c0325 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.677960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.678619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.678922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.679189 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.679863 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.681805 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.709136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1427/2864] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:38.713623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.726023 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381549e-baf2-4d00-afd0-947aa175c5ef None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.788917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.789823 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.790210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.790606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.791738 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.797604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.817930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1438/2865] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:38 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:38.822023 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.832476 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18675f9-195a-4bad-94e8-9346cebb50a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.837690 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.838335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.838611 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.838917 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.839636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.841244 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.852100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1428/2866] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:38.856309 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.867493 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a33b6-5c02-463c-a403-ba2ad420c0dc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.872339 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.873321 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.873617 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.874011 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] PATH_INFO: `/v3/projects/10a1aadb91264fe29cd4e82ad551e66a/users/a815120638774668a43786fe4ba38a2c/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.890267 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a1aadb91264fe29cd4e82ad551e66a, user_id=a815120638774668a43786fe4ba38a2c, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.892983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.908094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1439/2867] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:38 2022] PUT /identity/v3/projects/10a1aadb91264fe29cd4e82ad551e66a/users/a815120638774668a43786fe4ba38a2c/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:38.912349 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:38.922634 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb5af-c29f-46d4-932f-f2d8fe30b39b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:38.985402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e818d-d349-468e-bc7a-a4e3c3f23394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:38.985977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e818d-d349-468e-bc7a-a4e3c3f23394 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:38.986257 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e818d-d349-468e-bc7a-a4e3c3f23394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:38.986526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e818d-d349-468e-bc7a-a4e3c3f23394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:38.987280 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e818d-d349-468e-bc7a-a4e3c3f23394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:38.988852 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e818d-d349-468e-bc7a-a4e3c3f23394 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:38.999240 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1429/2868] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:24:39.004502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:39.022265 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdca8d-c330-4347-9be3-066cec9f9c3c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:39.031197 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:39.032499 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:39.032868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:39.033212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] PATH_INFO: `/v3/projects/10a1aadb91264fe29cd4e82ad551e66a/users/a815120638774668a43786fe4ba38a2c/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:39.036582 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a1aadb91264fe29cd4e82ad551e66a, user_id=a815120638774668a43786fe4ba38a2c, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:39.039592 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:39.051811 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1440/2869] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:39 2022] PUT /identity/v3/projects/10a1aadb91264fe29cd4e82ad551e66a/users/a815120638774668a43786fe4ba38a2c/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:39.062073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:39.062328 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:39.062600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:39.090577 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] MFA Rules not processed for user `a815120638774668a43786fe4ba38a2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:39.091827 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:39.173936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1430/2870] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:39.185937 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:39.188066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:39.202795 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:39.250451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a55f4-6169-4239-81ad-e7ee329038df admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:39.255047 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:39.255694 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:39.255968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:39.256222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:39.265787 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:39.267278 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:39.276902 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1441/2871] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:39.364986 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:39.374997 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748da8a7-30d8-470d-a224-1b9b913bb83b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:39.378786 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b0898-2fa6-4051-ae56-c593af5c9b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:39.379463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0898-2fa6-4051-ae56-c593af5c9b46 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:39.379748 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0898-2fa6-4051-ae56-c593af5c9b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:39.380021 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0898-2fa6-4051-ae56-c593af5c9b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:39.404791 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:39.441599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b0898-2fa6-4051-ae56-c593af5c9b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:39.443383 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b0898-2fa6-4051-ae56-c593af5c9b46 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:39.453104 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1431/2872] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:39.729552 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:39.791935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67d153-5ce0-470c-bc32-5f2743162374 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:39.796812 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:39.797698 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:39.797978 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:39.798254 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] PATH_INFO: `/v3/users/ecf42a27073243fe94fd739ceb5a72b5` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:39.799554 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf42a27073243fe94fd739ceb5a72b5)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:39.801525 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:39.854446 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf42a27073243fe94fd739ceb5a72b5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:39.855327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf42a27073243fe94fd739ceb5a72b5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:39.856168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf42a27073243fe94fd739ceb5a72b5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:39.873791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf42a27073243fe94fd739ceb5a72b5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:39.882353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf42a27073243fe94fd739ceb5a72b5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:39.898053 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1442/2873] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:39 2022] DELETE /identity/v3/users/ecf42a27073243fe94fd739ceb5a72b5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:39.902925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:39.903185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:39.903508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:39.921123 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:39.922358 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:39.984386 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1432/2874] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:24:39.994935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:39.996465 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:40.039182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:40.067718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc69f5-e42f-496f-a996-1736248a5c7e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:40.071735 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:40.072406 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:40.072688 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:40.072961 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:40.082556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:40.084062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:40.093999 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1443/2875] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:40.369884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:40.371432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:40.383921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531142c8-995f-47ec-a0fc-38d5c735597c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:40.388996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:40.389840 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:40.390119 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:40.390397 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] PATH_INFO: `/v3/projects/dbd0ae6599d64380b9646f488fa437f5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:40.391651 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd0ae6599d64380b9646f488fa437f5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:40.393666 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:40.443100 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd0ae6599d64380b9646f488fa437f5', 'request_id': 'req-964870db-f449-42f2-90e3-0bcf36394f90'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.449392 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd0ae6599d64380b9646f488fa437f5', 'request_id': 'req-964870db-f449-42f2-90e3-0bcf36394f90'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.456109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd0ae6599d64380b9646f488fa437f5', 'request_id': 'req-964870db-f449-42f2-90e3-0bcf36394f90'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.472131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] The token cache is being invalidate because project dbd0ae6599d64380b9646f488fa437f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:40.472401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.473321 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.475778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1433/2876] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:40 2022] DELETE /identity/v3/projects/dbd0ae6599d64380b9646f488fa437f5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:40.479935 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:40.489763 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b4aa7-4010-4956-b7e5-0eccad9e6387 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:40.549802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:40.550674 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:40.550983 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:40.551255 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] PATH_INFO: `/v3/users/a815120638774668a43786fe4ba38a2c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:40.552473 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a815120638774668a43786fe4ba38a2c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:40.554415 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:40.603404 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a815120638774668a43786fe4ba38a2c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.604063 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a815120638774668a43786fe4ba38a2c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.604658 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a815120638774668a43786fe4ba38a2c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.617438 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a815120638774668a43786fe4ba38a2c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.624897 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-98a2194a-9ae6-4437-b4ef-dec173576033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a815120638774668a43786fe4ba38a2c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:40.639866 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1444/2877] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:40 2022] DELETE /identity/v3/users/a815120638774668a43786fe4ba38a2c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:40.926353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:40.928627 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:40.942516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964870db-f449-42f2-90e3-0bcf36394f90 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:40.989451 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:41.002431 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.003330 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.003609 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.003877 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] PATH_INFO: `/v3/projects/10a1aadb91264fe29cd4e82ad551e66a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.005194 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a1aadb91264fe29cd4e82ad551e66a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.007251 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.056534 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a1aadb91264fe29cd4e82ad551e66a', 'request_id': 'req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:41.063069 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a1aadb91264fe29cd4e82ad551e66a', 'request_id': 'req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:41.070436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a1aadb91264fe29cd4e82ad551e66a', 'request_id': 'req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:41.084628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] The token cache is being invalidate because project 10a1aadb91264fe29cd4e82ad551e66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:41.084922 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:41.085598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e0ebf68c-58cb-4f27-8dbb-4a5580190c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:41.087573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1434/2878] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:40 2022] DELETE /identity/v3/projects/10a1aadb91264fe29cd4e82ad551e66a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:41.094457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.094941 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.095031 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.114449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:41.115709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:41.169407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:41.182099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1445/2879] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:41.190159 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.190440 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.190708 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.212294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:41.213502 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:41.222518 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1435/2880] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:41.227235 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.242208 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f0956-03c7-43db-ae1c-bb2800c461be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.248882 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.249568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.249850 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.250107 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.250872 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:24:41.251221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.253330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.270013 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1446/2881] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:24:41.276636 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.290934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83272d2-dc6f-4fa2-ad32-361e05ebdb54 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.296948 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.297596 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.297875 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.298150 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.298988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.300908 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.329436 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1436/2882] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:41 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:24:41.334002 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.347021 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934222c-0602-43cc-9236-673f4bdd8d6a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.419844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.420432 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.420710 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.420982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.422052 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.424018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.452078 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1447/2883] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:41 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:41.454683 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.469646 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8cf40-f786-49ba-846d-41d3268216f8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.476143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.477013 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.477326 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.477648 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.478729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.480873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.496052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1437/2884] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:41.500246 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.510788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7768e1-8c16-463e-b08b-88020b98d1d1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.516416 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650316f-e865-4c7e-9529-561df186372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.517119 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650316f-e865-4c7e-9529-561df186372a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.517399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650316f-e865-4c7e-9529-561df186372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.517682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650316f-e865-4c7e-9529-561df186372a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.518427 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650316f-e865-4c7e-9529-561df186372a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.520062 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650316f-e865-4c7e-9529-561df186372a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.530419 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1448/2885] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:41.534485 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.545112 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0182d-357d-4d7c-b107-9bf2fc2e7ae6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.550702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.551600 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.551869 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.552131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] PATH_INFO: `/v3/projects/a14e8d1c6bf346d79f8c27f979d33288/users/909e936f08a748f2b1766c0b33e75129/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.568130 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14e8d1c6bf346d79f8c27f979d33288, user_id=909e936f08a748f2b1766c0b33e75129, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.570861 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.586656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1438/2886] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:41 2022] PUT /identity/v3/projects/a14e8d1c6bf346d79f8c27f979d33288/users/909e936f08a748f2b1766c0b33e75129/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:24:41.593222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a72a3-5205-4932-b558-583ed948c0c3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.593568 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a72a3-5205-4932-b558-583ed948c0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.593930 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a72a3-5205-4932-b558-583ed948c0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.616320 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a53a72a3-5205-4932-b558-583ed948c0c3 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:41.617553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a53a72a3-5205-4932-b558-583ed948c0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:41.696761 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1449/2887] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:41.708530 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:41.710737 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:41.777212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b48b-3b38-4316-b331-7130271ed4b2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:41.781619 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:41.782275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:41.782556 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:41.782858 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:41.793899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:41.795384 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:41.805510 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1439/2888] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:42.462893 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.463420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.463420 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.492147 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] MFA Rules not processed for user `909e936f08a748f2b1766c0b33e75129`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:42.493388 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:42.542626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:42.561112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1450/2889] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:42.565555 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:42.575437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12b612-82da-4e12-a3b8-da70f11633e5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:42.580143 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:42.580849 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.581192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.581528 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.582294 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:42.584287 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:42.599192 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:42.613745 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1440/2890] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:42 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:42.618209 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:42.630137 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4c45e-9830-4280-bbf8-4910468d5743 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:42.691212 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:42.691887 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.692168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.692440 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.693160 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:42.695281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:42.717317 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1451/2891] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:42 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:24:42.721452 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:42.732203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ec6a1-69dd-4c00-9cae-d7609d7bf08d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:42.736955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:42.737632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.737910 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.738177 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.738955 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:42.740553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:42.751362 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1441/2892] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:42.755564 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:42.765682 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18863103-c8fe-49c0-8e96-4ded679ccbfe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:42.770932 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:42.771813 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.772100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.772369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] PATH_INFO: `/v3/projects/6191f7601c974a72a3e25c1cb09deccd/users/893c711386bd4845b07d92407c26a85e/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.787296 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6191f7601c974a72a3e25c1cb09deccd, user_id=893c711386bd4845b07d92407c26a85e, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:42.790498 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:42.805164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1452/2893] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:42 2022] PUT /identity/v3/projects/6191f7601c974a72a3e25c1cb09deccd/users/893c711386bd4845b07d92407c26a85e/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:42.809302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:42.819388 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9743792-8729-490d-8c9b-cf06d49fb89a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:42.876770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee5e55-27d1-4c4d-b876-d5c9828025d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:42.877441 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee5e55-27d1-4c4d-b876-d5c9828025d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.877723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee5e55-27d1-4c4d-b876-d5c9828025d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.877967 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee5e55-27d1-4c4d-b876-d5c9828025d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.878702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee5e55-27d1-4c4d-b876-d5c9828025d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:42.880309 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee5e55-27d1-4c4d-b876-d5c9828025d0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:42.890544 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1442/2894] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:42.896403 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:42.908346 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278eba58-1657-463c-9f16-35b215041928 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:42.915069 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:42.915957 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:42.916231 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:42.916500 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] PATH_INFO: `/v3/projects/6191f7601c974a72a3e25c1cb09deccd/users/893c711386bd4845b07d92407c26a85e/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:42.918695 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] RBAC: Authorizing `identity:create_grant(project_id=6191f7601c974a72a3e25c1cb09deccd, user_id=893c711386bd4845b07d92407c26a85e, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:42.921421 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:42.931721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1453/2895] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:42 2022] PUT /identity/v3/projects/6191f7601c974a72a3e25c1cb09deccd/users/893c711386bd4845b07d92407c26a85e/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:43.591976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:43.592503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:43.592503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:43.620191 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] MFA Rules not processed for user `893c711386bd4845b07d92407c26a85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:43.621408 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:43.693736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1443/2896] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:43.704868 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:43.706642 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:43.772463 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5b2e-6452-4b53-8fa4-0a40a21feace admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:43.776729 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:43.777386 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:43.777662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:43.777936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:43.836110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:43.837926 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:43.841641 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:43.848533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1454/2897] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:45.223971 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:45.225559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:45.244113 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483602-41c0-43c8-8a10-91a173d460c7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:45.249223 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:45.249868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:45.250141 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:45.250411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:45.261437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:45.262892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:45.272508 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1444/2898] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:24:53.851533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:54.308702 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:54.310398 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:54.373524 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cecd78-ec79-4119-8193-449265444e47 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:54.377275 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:54.378048 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:54.378169 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:54.378469 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:54.380366 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:54.389997 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:54.391479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:54.399669 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1455/2899] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:24:55.268406 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=76 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:58.424823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:58.489091 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649de6c0-4ac5-45bc-895b-a2245affb941 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:58.494187 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:58.495136 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:58.495407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:58.495674 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] PATH_INFO: `/v3/users/909e936f08a748f2b1766c0b33e75129` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:58.496865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] RBAC: Authorizing `identity:delete_user(user_id=909e936f08a748f2b1766c0b33e75129)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:58.498784 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:58.549493 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '909e936f08a748f2b1766c0b33e75129'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.550152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '909e936f08a748f2b1766c0b33e75129'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.557134 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '909e936f08a748f2b1766c0b33e75129'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.557713 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '909e936f08a748f2b1766c0b33e75129'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.570056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '909e936f08a748f2b1766c0b33e75129'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.588771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1445/2900] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:58 2022] DELETE /identity/v3/users/909e936f08a748f2b1766c0b33e75129 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:58.778599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:58.790177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939afa9-8557-409e-84bf-bf36e4df3f06 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:58.852221 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:58.853090 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:58.853361 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:58.853597 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] PATH_INFO: `/v3/projects/a14e8d1c6bf346d79f8c27f979d33288` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:58.854871 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14e8d1c6bf346d79f8c27f979d33288)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:58.856881 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:58.907274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14e8d1c6bf346d79f8c27f979d33288', 'request_id': 'req-f52744f2-bcda-48cf-9e66-1440ca152e04'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.913513 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14e8d1c6bf346d79f8c27f979d33288', 'request_id': 'req-f52744f2-bcda-48cf-9e66-1440ca152e04'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.919715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14e8d1c6bf346d79f8c27f979d33288', 'request_id': 'req-f52744f2-bcda-48cf-9e66-1440ca152e04'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.933350 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] The token cache is being invalidate because project a14e8d1c6bf346d79f8c27f979d33288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:58.933637 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.934409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:58.936112 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1456/2901] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:58 2022] DELETE /identity/v3/projects/a14e8d1c6bf346d79f8c27f979d33288 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:58.940926 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:58.942411 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:58.955689 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b2ebe-7298-442a-a49e-97c1e50e762c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.013702 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.014579 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.014888 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.015185 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] PATH_INFO: `/v3/users/893c711386bd4845b07d92407c26a85e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.016405 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] RBAC: Authorizing `identity:delete_user(user_id=893c711386bd4845b07d92407c26a85e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.018315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.039649 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:59.079723 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893c711386bd4845b07d92407c26a85e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.080382 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893c711386bd4845b07d92407c26a85e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.087925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893c711386bd4845b07d92407c26a85e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.088608 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893c711386bd4845b07d92407c26a85e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.101448 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-171d7f22-dc70-42e5-9a13-13f58292881e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893c711386bd4845b07d92407c26a85e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.115744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1446/2902] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:24:58 2022] DELETE /identity/v3/users/893c711386bd4845b07d92407c26a85e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:59.296222 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.297858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:59.313308 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52744f2-bcda-48cf-9e66-1440ca152e04 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.345218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:59.380906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.381791 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.382066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.382330 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] PATH_INFO: `/v3/projects/6191f7601c974a72a3e25c1cb09deccd` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.383534 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6191f7601c974a72a3e25c1cb09deccd)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.385807 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.438273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6191f7601c974a72a3e25c1cb09deccd', 'request_id': 'req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.444714 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6191f7601c974a72a3e25c1cb09deccd', 'request_id': 'req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.450799 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6191f7601c974a72a3e25c1cb09deccd', 'request_id': 'req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.466094 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] The token cache is being invalidate because project 6191f7601c974a72a3e25c1cb09deccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:24:59.466461 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.467104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ddd292a-3118-4c48-b59c-4b6269ea3c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:24:59.468653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1457/2903] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:24:59 2022] DELETE /identity/v3/projects/6191f7601c974a72a3e25c1cb09deccd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:59.473722 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.474206 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.474298 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.492952 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:59.494149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:59.553271 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1447/2904] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:59.562211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.562494 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.562790 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.581697 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:59.582944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:24:59.592839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1458/2905] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:24:59.598286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.608195 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6288c75-3177-4b47-a3c3-dbf19d1306b8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.613823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.614474 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.614782 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.615063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.615776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:24:59.616079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.617682 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.629873 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1448/2906] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:24:59.634374 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.643952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf057b9f-d158-47c2-af70-010b1f376b00 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.649996 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.650653 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.651110 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.651574 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.652785 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.655954 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.687042 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1459/2907] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:24:59 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:59.691367 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.704285 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad15f-c556-46c0-a09b-e37241ec7bc4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.765115 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.765778 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.766056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.766329 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.767074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.768977 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.790960 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1449/2908] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:24:59 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:24:59.795433 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.806342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca3e6c-7e0c-4e6d-ac40-77d7dc4bcd26 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.811968 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.812631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.812909 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.813182 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.813905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.815586 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.827576 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1460/2909] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:24:59.831909 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.842667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc6b7-83ef-41ab-954c-da5de5d05a50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.847710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a474a57-ec4d-430f-943a-b8fb69a86572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.848463 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a474a57-ec4d-430f-943a-b8fb69a86572 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.848647 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a474a57-ec4d-430f-943a-b8fb69a86572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.848941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a474a57-ec4d-430f-943a-b8fb69a86572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.849687 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a474a57-ec4d-430f-943a-b8fb69a86572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.851275 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a474a57-ec4d-430f-943a-b8fb69a86572 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.861993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1450/2910] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:24:59.866712 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:24:59.882210 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a40d2b-ebd4-40cb-bc51-151e7f906ef8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:24:59.888776 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:24:59.889655 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.889928 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.890194 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] PATH_INFO: `/v3/projects/fd26cf8f68a84e438688f763a26d8bda/users/589166bf125141fdb25f303b34284845/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.906915 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd26cf8f68a84e438688f763a26d8bda, user_id=589166bf125141fdb25f303b34284845, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:24:59.909685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:24:59.925249 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1461/2911] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:24:59 2022] PUT /identity/v3/projects/fd26cf8f68a84e438688f763a26d8bda/users/589166bf125141fdb25f303b34284845/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:24:59.935548 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:24:59.935837 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:24:59.936137 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:24:59.943064 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:24:59.964019 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] MFA Rules not processed for user `589166bf125141fdb25f303b34284845`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:24:59.965248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:00.031821 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1451/2912] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:24:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:25:00.036630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.100019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320d85c-0e41-45b2-8e1d-b7c60bb646ec admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.104838 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.105545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.105773 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.106072 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.106788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.108706 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.112097 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:00.141599 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1462/2913] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:25:00 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:25:00.147335 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.160060 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3f9f1-1e0c-4c24-af09-1983c0920ca3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.219936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.220607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.220889 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.221161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.221874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.223902 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.244938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1452/2914] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:25:00 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:25:00.248975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.259670 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fb698-e17a-477d-b007-69a6f9c7a71b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.265516 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.266218 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.266514 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.266819 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.267537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.269175 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.280281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1463/2915] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:25:00.286978 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.303950 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe87e6d-ca84-4fc0-a5b1-f7aecf82e82e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.310621 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.311551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.311830 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.312102 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] PATH_INFO: `/v3/projects/92c49c70c4d24efa9329947454729794/users/3938d11b92fa46158774254467aec6f6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.327721 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c49c70c4d24efa9329947454729794, user_id=3938d11b92fa46158774254467aec6f6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.330432 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.345893 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1453/2916] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:25:00 2022] PUT /identity/v3/projects/92c49c70c4d24efa9329947454729794/users/3938d11b92fa46158774254467aec6f6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:00.349876 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.360282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1365e4a-429f-4dba-b686-37995af8ec78 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.417501 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ceb57-dad6-40c5-8eb0-be927de6913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.418190 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceb57-dad6-40c5-8eb0-be927de6913e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.418468 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceb57-dad6-40c5-8eb0-be927de6913e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.418722 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceb57-dad6-40c5-8eb0-be927de6913e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.419510 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ceb57-dad6-40c5-8eb0-be927de6913e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.421124 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ceb57-dad6-40c5-8eb0-be927de6913e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.431252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1464/2917] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:25:00.435656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.446353 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16ed03-50b8-4e2a-bda4-f3d508f35ffd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.451994 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.452868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.453148 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.453420 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] PATH_INFO: `/v3/projects/92c49c70c4d24efa9329947454729794/users/3938d11b92fa46158774254467aec6f6/roles/fa7c01e623754dfea0b39b9ee8f0d51d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.455653 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c49c70c4d24efa9329947454729794, user_id=3938d11b92fa46158774254467aec6f6, role_id=fa7c01e623754dfea0b39b9ee8f0d51d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.458293 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.469266 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1454/2918] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:25:00 2022] PUT /identity/v3/projects/92c49c70c4d24efa9329947454729794/users/3938d11b92fa46158774254467aec6f6/roles/fa7c01e623754dfea0b39b9ee8f0d51d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:00.479602 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.479912 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.480204 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.506711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] MFA Rules not processed for user `3938d11b92fa46158774254467aec6f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:25:00.507952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:00.579703 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1465/2919] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:25:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:25:00.591222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:00.593317 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:00.655871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:00.662248 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1cf21-f82c-4ac8-8790-00a5344ac6af admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:00.667315 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523ce17-71cc-40a5-83ac-e9eac46e7182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:00.667962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523ce17-71cc-40a5-83ac-e9eac46e7182 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:00.668242 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523ce17-71cc-40a5-83ac-e9eac46e7182 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:00.668511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523ce17-71cc-40a5-83ac-e9eac46e7182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:00.727193 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523ce17-71cc-40a5-83ac-e9eac46e7182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:00.728625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523ce17-71cc-40a5-83ac-e9eac46e7182 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:00.738161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1455/2920] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:25:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:25:03.278917 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:03.337708 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:03.344271 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b505bb-f4a1-46ef-bb34-fcef1a1bb215 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:03.349392 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:03.350270 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:03.350553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:03.350839 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] PATH_INFO: `/v3/users/589166bf125141fdb25f303b34284845` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:03.352036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=589166bf125141fdb25f303b34284845)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:03.353975 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:03.411327 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589166bf125141fdb25f303b34284845'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:03.412010 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589166bf125141fdb25f303b34284845'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:03.412673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589166bf125141fdb25f303b34284845'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:03.425981 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589166bf125141fdb25f303b34284845'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:03.434186 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589166bf125141fdb25f303b34284845'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:03.450371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1466/2921] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:25:03 2022] DELETE /identity/v3/users/589166bf125141fdb25f303b34284845 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:03.455904 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:03.456377 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:03.456494 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:03.475173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:25:03.476398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:03.538074 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1456/2922] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:25:03.548407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:03.549991 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:03.613604 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaedf3-60b4-4884-8827-3e803d9be793 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:03.618150 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:03.618817 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:03.619104 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:03.619381 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:03.628952 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:03.630435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:03.640158 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1467/2923] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:25:03.938131 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:03.939656 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:03.953490 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76b5b7-f3e3-4240-85bc-914b23b81a68 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:03.959819 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:03.960941 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:03.961161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:03.961425 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] PATH_INFO: `/v3/projects/fd26cf8f68a84e438688f763a26d8bda` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:03.963154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd26cf8f68a84e438688f763a26d8bda)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:03.965063 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:04.015599 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd26cf8f68a84e438688f763a26d8bda', 'request_id': 'req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.021857 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd26cf8f68a84e438688f763a26d8bda', 'request_id': 'req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.027511 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd26cf8f68a84e438688f763a26d8bda', 'request_id': 'req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.041026 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] The token cache is being invalidate because project fd26cf8f68a84e438688f763a26d8bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:25:04.041252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.041879 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.043568 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1457/2924] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:25:03 2022] DELETE /identity/v3/projects/fd26cf8f68a84e438688f763a26d8bda => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:04.047699 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:04.057151 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7b602-be75-455e-9b60-bdf95a17d2ea service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:04.114245 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:04.115133 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:04.115407 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:04.115676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] PATH_INFO: `/v3/users/3938d11b92fa46158774254467aec6f6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:04.116883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3938d11b92fa46158774254467aec6f6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:04.118862 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:04.138895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:04.166359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3938d11b92fa46158774254467aec6f6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.167056 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3938d11b92fa46158774254467aec6f6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.167643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3938d11b92fa46158774254467aec6f6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.179015 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3938d11b92fa46158774254467aec6f6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.185730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cc763a4a-22a3-482f-80a3-9d451fd59816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3938d11b92fa46158774254467aec6f6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.199114 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1468/2925] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:25:04 2022] DELETE /identity/v3/users/3938d11b92fa46158774254467aec6f6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:04.482445 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:04.484899 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:04.507806 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62dd1d-caf2-454f-ae59-c2e74a6382d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:04.533773 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:04.588153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:04.588996 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:04.589269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:04.589533 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] PATH_INFO: `/v3/projects/92c49c70c4d24efa9329947454729794` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:04.590779 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c49c70c4d24efa9329947454729794)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:04.592740 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:04.652338 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c49c70c4d24efa9329947454729794', 'request_id': 'req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.661459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c49c70c4d24efa9329947454729794', 'request_id': 'req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.667679 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c49c70c4d24efa9329947454729794', 'request_id': 'req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.684025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] The token cache is being invalidate because project 92c49c70c4d24efa9329947454729794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:25:04.684334 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.685038 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-007ecb0b-5a6b-4f4c-a2bd-d3583f81f598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:04.686612 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1458/2926] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:25:04 2022] DELETE /identity/v3/projects/92c49c70c4d24efa9329947454729794 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:04.692778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:04.693281 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:04.693384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:04.720373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:25:04.721964 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:04.786556 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1469/2927] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:25:04.795655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:04.795936 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:04.796204 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:04.815272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:25:04.816602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:04.824993 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1459/2928] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:25:04.829449 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:04.840282 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8e673-c8e0-401f-812b-ff46f9c44ec1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:04.847802 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:04.848476 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:04.848755 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:04.849030 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:04.849747 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:25:04.850046 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:04.851743 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:04.866057 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1470/2929] 10.222.0.32 () {58 vars in 1152 bytes} [Wed May 25 15:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:25:04.873934 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:04.886034 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c0cc9-cc0e-4b83-aa93-d0eef9a725fa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:04.894053 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:04.894803 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:04.895088 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:04.895344 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:04.896288 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:04.898729 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:04.928710 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1460/2930] 10.222.0.32 () {60 vars in 1152 bytes} [Wed May 25 15:25:04 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:25:04.934496 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:04.946138 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d2079-999c-43f8-b860-ff13789aa00d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:05.011508 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:05.012219 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:05.012497 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:05.012818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:05.013509 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:05.015558 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:05.038543 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1471/2931] 10.222.0.32 () {60 vars in 1143 bytes} [Wed May 25 15:25:04 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:25:05.042796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:05.053641 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d554-3392-4329-b321-c66026075ffd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:05.059563 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:05.060222 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:05.060503 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:05.060759 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:05.061477 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:05.063109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:05.074726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1461/2932] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:25:05.078856 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:05.091164 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d436261-423f-4278-b82b-68a50e21fd84 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:05.096402 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc9c8b-388d-40bb-9ccf-d3c74ca0f769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:05.097073 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc9c8b-388d-40bb-9ccf-d3c74ca0f769 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:05.097358 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc9c8b-388d-40bb-9ccf-d3c74ca0f769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:05.097622 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc9c8b-388d-40bb-9ccf-d3c74ca0f769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:05.098368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc9c8b-388d-40bb-9ccf-d3c74ca0f769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:05.100022 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc9c8b-388d-40bb-9ccf-d3c74ca0f769 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:05.111226 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1472/2933] 10.222.0.32 () {58 vars in 1121 bytes} [Wed May 25 15:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:25:05.115831 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:05.126453 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731769a0-f9a6-4719-9dd1-7477d8032223 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:05.132067 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:05.133003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:05.133282 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:05.133551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] PATH_INFO: `/v3/projects/4407435ad17a4c6aa56e6fc79038caf6/users/c71a8070785d463b97ada466cf114f3d/roles/3e228f026e734b8b9c06f578f2012163` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:05.148631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4407435ad17a4c6aa56e6fc79038caf6, user_id=c71a8070785d463b97ada466cf114f3d, role_id=3e228f026e734b8b9c06f578f2012163)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:05.151319 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:05.169516 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1462/2934] 10.222.0.32 () {60 vars in 1482 bytes} [Wed May 25 15:25:05 2022] PUT /identity/v3/projects/4407435ad17a4c6aa56e6fc79038caf6/users/c71a8070785d463b97ada466cf114f3d/roles/3e228f026e734b8b9c06f578f2012163 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:05.180041 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:05.180401 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:05.180673 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:05.187369 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:05.213863 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] MFA Rules not processed for user `c71a8070785d463b97ada466cf114f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:25:05.215168 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:05.291066 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1473/2935] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:25:05.302340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:05.304407 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:05.367456 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bac74-1d9c-4c01-a2d6-a31087d75252 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:05.372664 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c5d23-aee1-4cd8-ab4c-ad8596ee9234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:05.373307 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5d23-aee1-4cd8-ab4c-ad8596ee9234 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:05.373605 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5d23-aee1-4cd8-ab4c-ad8596ee9234 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:05.373871 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5d23-aee1-4cd8-ab4c-ad8596ee9234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:05.379631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:05.384540 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c5d23-aee1-4cd8-ab4c-ad8596ee9234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:05.386149 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c5d23-aee1-4cd8-ab4c-ad8596ee9234 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:05.396214 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1463/2936] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:25:15.287936 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:15.398085 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:15.528088 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:15.598734 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ba95-6595-400f-a5f1-5b2a306ac357 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:15.604043 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:15.604923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:15.605192 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:15.605457 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] PATH_INFO: `/v3/users/c71a8070785d463b97ada466cf114f3d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:15.606667 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71a8070785d463b97ada466cf114f3d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:15.608636 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:15.668542 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71a8070785d463b97ada466cf114f3d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:15.669332 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71a8070785d463b97ada466cf114f3d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:15.670020 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71a8070785d463b97ada466cf114f3d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:15.684185 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71a8070785d463b97ada466cf114f3d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:15.691898 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71a8070785d463b97ada466cf114f3d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:15.709579 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1474/2937] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:25:15 2022] DELETE /identity/v3/users/c71a8070785d463b97ada466cf114f3d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:25:15.715459 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:15.715780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:15.716095 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:15.735551 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] MFA Rules not processed for user `73561edcff5e424d9ae49ab57406b79a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:25:15.736785 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:15.801814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1464/2938] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:25:15.812004 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:15.813635 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:15.880170 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6dab0-91f8-4e95-8cd0-ee4c44937f03 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:15.884372 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:15.885071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:15.885349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:15.885616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:15.896259 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:15.897816 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:15.909373 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1475/2939] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:25:16.191268 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:16.192892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:16.208667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ae603-ca7a-462c-aa6f-638e06467361 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:16.214078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:16.214991 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:16.215314 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:16.215629 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] PATH_INFO: `/v3/projects/4407435ad17a4c6aa56e6fc79038caf6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:16.216744 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4407435ad17a4c6aa56e6fc79038caf6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:16.218672 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:16.271006 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4407435ad17a4c6aa56e6fc79038caf6', 'request_id': 'req-bb37a03a-d0f1-471f-9640-236720dd0b20'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:16.277398 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4407435ad17a4c6aa56e6fc79038caf6', 'request_id': 'req-bb37a03a-d0f1-471f-9640-236720dd0b20'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:16.283079 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4407435ad17a4c6aa56e6fc79038caf6', 'request_id': 'req-bb37a03a-d0f1-471f-9640-236720dd0b20'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:16.297302 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] The token cache is being invalidate because project 4407435ad17a4c6aa56e6fc79038caf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:25:16.297592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:16.298260 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:25:16.299938 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1465/2940] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:25:16 2022] DELETE /identity/v3/projects/4407435ad17a4c6aa56e6fc79038caf6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:25:25.904099 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:26.294592 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:39.665804 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:39.667651 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:39.748225 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35759081-6a1d-46e4-ba11-20a6fae94c37 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:39.810083 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:39.810742 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:39.811049 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:39.811324 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:39.813280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:39.824778 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:39.826360 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833cb79-81a2-4f32-81ce-91a4a6d2d414 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:39.836371 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1476/2941] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:25:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:25:48.472361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:25:48.473866 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:48.534212 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37a03a-d0f1-471f-9640-236720dd0b20 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:25:48.538139 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:25:48.538812 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:25:48.539122 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:25:48.539396 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:25:48.541316 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:25:48.607632 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:25:48.609056 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:25:48.618092 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1466/2942] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:25:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:25:49.831905 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=58 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:25:58.614146 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=48 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:26:00.731972 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:00.732530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:00.732631 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:00.760200 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] MFA Rules not processed for user `3db86b8d51074eec9ea8a10cacb8a602`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:26:00.762399 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:00.773503 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1477/2943] 10.222.0.32 () {58 vars in 957 bytes} [Wed May 25 15:26:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:26:00.814230 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:00.815814 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:00.829008 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753ea75-0075-47f7-b1f2-8b470579bae1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:00.832962 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc3c334dc46b47a3b8e415e913c29249', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:00.834047 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:00.834487 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:00.834892 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:00.837764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:00.941312 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:00.942726 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:00.954764 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1467/2944] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:26:01.933603 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:01.935102 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:01.944868 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:01.947305 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:01.950869 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0be80b-1aa9-4efc-899c-34b085d7fff5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:01.956263 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:01.957012 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:01.957359 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:01.957662 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:01.959435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:01.971883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:01.973502 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:01.983796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1478/2945] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:26:02.006573 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54320bda-8811-4c02-b98a-6be02b1ca1d4 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:02.010695 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:02.011378 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:02.011655 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:02.011921 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:02.014166 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:02.029364 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:02.030807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:02.042161 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1468/2946] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:26:11.979623 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=9 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:26:12.036835 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=63 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:26:16.232607 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:16.234858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:16.252873 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0fa84-6d99-4616-ae0a-f86a45f0ca3a service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:16.258861 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:16.259537 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:16.259815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:16.260100 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:16.261878 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:16.275211 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:16.277445 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:16.289750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1479/2947] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:26:21.193249 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:21.194807 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:21.205736 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245595d4-ecff-4d29-9af6-5969702e550a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:21.210472 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:21.211144 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:21.211418 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:21.211688 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:21.213602 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:21.282945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:21.284368 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5bb6de-5a41-41cb-a209-3f06465b4213 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:21.294631 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1469/2948] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:26:26.284982 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:26:31.289624 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=30 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:26:47.102193 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:47.112019 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b4659-e3b3-4885-b57d-1bd5a051ac93 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:47.116368 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:47.117001 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:47.117280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:47.117550 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:47.119620 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:47.130353 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:47.131818 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9fc14-b73f-450e-883f-4455cb0c8451 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:47.140451 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1480/2949] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:26:48.258965 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352357d-3b23-458b-a306-9bcb5db5adfb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:48.259576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352357d-3b23-458b-a306-9bcb5db5adfb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:48.259576 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352357d-3b23-458b-a306-9bcb5db5adfb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:48.260897 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1470/2950] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 15:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:26:48.267228 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8cfb9-ea51-4947-8408-40debcd1bbc8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:48.267724 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8cfb9-ea51-4947-8408-40debcd1bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:48.267825 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8cfb9-ea51-4947-8408-40debcd1bbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:48.309583 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-51d8cfb9-ea51-4947-8408-40debcd1bbc8 None None] MFA Rules not processed for user `db335e266cd14d45868e6875a9ea34d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:26:48.310796 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-51d8cfb9-ea51-4947-8408-40debcd1bbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:48.317923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1481/2951] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:26:48.330098 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352357d-3b23-458b-a306-9bcb5db5adfb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:48.341763 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352357d-3b23-458b-a306-9bcb5db5adfb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:48.346160 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca96cd2-0268-4419-a5a2-b3fe2807461c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:48.346886 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96cd2-0268-4419-a5a2-b3fe2807461c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:48.347278 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96cd2-0268-4419-a5a2-b3fe2807461c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:48.347692 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca96cd2-0268-4419-a5a2-b3fe2807461c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:48.356393 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96cd2-0268-4419-a5a2-b3fe2807461c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:48.357945 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca96cd2-0268-4419-a5a2-b3fe2807461c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:48.366005 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1471/2952] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:26:49.472274 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1d3d6-7357-4bc6-bb06-f1d6fd1ee754 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:49.472815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1d3d6-7357-4bc6-bb06-f1d6fd1ee754 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:49.472815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1d3d6-7357-4bc6-bb06-f1d6fd1ee754 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:49.475132 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1482/2953] 10.222.0.32 () {54 vars in 921 bytes} [Wed May 25 15:26:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:26:49.480607 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:49.481025 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:49.481269 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:49.522028 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] MFA Rules not processed for user `eeee4e4e0fc74d7db8b5cb9d0ba806a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:26:49.523299 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:26:49.533031 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1472/2954] 10.222.0.32 () {58 vars in 1018 bytes} [Wed May 25 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:26:49.544889 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1d3d6-7357-4bc6-bb06-f1d6fd1ee754 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:26:49.558794 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1d3d6-7357-4bc6-bb06-f1d6fd1ee754 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:26:49.562788 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:26:49.563435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:26:49.563685 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:26:49.563973 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:26:49.576093 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:26:49.577709 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:26:49.589018 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1483/2955] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:26:59.519029 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=15 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:26:59.584300 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=21 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:13.892626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:13.894409 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:13.909379 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde87862-2c5d-4241-b8c3-2b6ef10096b2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:13.915718 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:13.916677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:13.917165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:13.917626 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:13.919776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:13.934598 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:13.936352 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:13.948554 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1473/2956] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:28:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:28:23.944253 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:27.584081 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:27.604718 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6012254-165f-4ac3-ae67-f8f4fe2402e9 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:27.609435 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:27.610086 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:27.610363 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:27.610629 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:27.612774 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:27.701775 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:27.703271 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:27.722736 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1484/2957] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:28:30.591120 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:30.592667 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:30.635816 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e8e8-7282-429e-a2fc-f3a48c26fb74 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:30.647801 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:30.648342 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:30.648638 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:30.649003 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:30.652254 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:30.678215 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:30.679976 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:30.695308 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1474/2958] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:28:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:28:33.824994 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:33.827575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:33.839945 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f51a0-04c8-4041-8d18-6f9c835b53c3 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:33.848643 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:33.849539 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:33.849840 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:33.850149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] PATH_INFO: `/v3/users/3db86b8d51074eec9ea8a10cacb8a602` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:33.851384 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db86b8d51074eec9ea8a10cacb8a602)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:33.853342 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:33.902166 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db86b8d51074eec9ea8a10cacb8a602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:33.902906 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db86b8d51074eec9ea8a10cacb8a602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:33.903545 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db86b8d51074eec9ea8a10cacb8a602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:33.915130 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db86b8d51074eec9ea8a10cacb8a602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:33.921883 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db86b8d51074eec9ea8a10cacb8a602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:33.937777 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1485/2959] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:28:33 2022] DELETE /identity/v3/users/3db86b8d51074eec9ea8a10cacb8a602 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:28:34.132541 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:34.144770 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e6e9b-e735-476f-90c7-94d41513eab2 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:34.216363 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:34.217272 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:34.217553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:34.217823 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] PATH_INFO: `/v3/projects/88dfcfb36400474aa945f747610c5ad2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:34.219068 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dfcfb36400474aa945f747610c5ad2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:34.221300 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:34.271661 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dfcfb36400474aa945f747610c5ad2', 'request_id': 'req-54a448fa-1430-4872-95cf-e67bae50ed00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.278073 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dfcfb36400474aa945f747610c5ad2', 'request_id': 'req-54a448fa-1430-4872-95cf-e67bae50ed00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.284052 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dfcfb36400474aa945f747610c5ad2', 'request_id': 'req-54a448fa-1430-4872-95cf-e67bae50ed00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.297860 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] The token cache is being invalidate because project 88dfcfb36400474aa945f747610c5ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:28:34.298152 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.298796 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.300460 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1475/2960] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:28:34 2022] DELETE /identity/v3/projects/88dfcfb36400474aa945f747610c5ad2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:28:34.304606 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:34.306064 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:34.319844 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8dec-9c79-45cf-84d0-573aa7b89590 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:34.346294 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:34.379279 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:34.380177 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:34.380456 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:34.380730 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] PATH_INFO: `/v3/users/05d5004da6b14a26843a55faa346c6cd` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:34.381923 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d5004da6b14a26843a55faa346c6cd)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:34.383955 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:34.435014 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d5004da6b14a26843a55faa346c6cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.435676 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d5004da6b14a26843a55faa346c6cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.436287 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d5004da6b14a26843a55faa346c6cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.448383 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d5004da6b14a26843a55faa346c6cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.458409 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d5004da6b14a26843a55faa346c6cd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.476549 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1486/2961] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:28:34 2022] DELETE /identity/v3/users/05d5004da6b14a26843a55faa346c6cd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:28:34.657005 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:34.658693 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:34.673173 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a448fa-1430-4872-95cf-e67bae50ed00 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:34.734078 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:34.735007 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:34.735286 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:34.735559 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] PATH_INFO: `/v3/projects/3d2eaffc522042d981e4496050bf71ed` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:34.736776 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2eaffc522042d981e4496050bf71ed)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:34.738694 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:34.746628 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:34.788826 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2eaffc522042d981e4496050bf71ed', 'request_id': 'req-a4986573-37c4-4b42-b609-6e21b3574c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.794974 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2eaffc522042d981e4496050bf71ed', 'request_id': 'req-a4986573-37c4-4b42-b609-6e21b3574c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.800606 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2eaffc522042d981e4496050bf71ed', 'request_id': 'req-a4986573-37c4-4b42-b609-6e21b3574c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.814581 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] The token cache is being invalidate because project 3d2eaffc522042d981e4496050bf71ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:28:34.814907 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.815517 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:34.817055 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1476/2962] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:28:34 2022] DELETE /identity/v3/projects/3d2eaffc522042d981e4496050bf71ed => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:28:44.472750 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:44.811616 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:47.054252 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:47.055865 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:47.118237 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193bae3-b8b6-4b39-91a2-17436ae58ebf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:47.167616 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:47.168273 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:47.168553 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:47.168826 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:47.170719 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:47.232032 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:47.233512 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:47.242677 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1487/2963] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:28:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:28:50.874677 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:50.876401 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:50.982560 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986573-37c4-4b42-b609-6e21b3574c60 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:50.988270 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeee4e4e0fc74d7db8b5cb9d0ba806a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:50.988925 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:50.989210 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:50.989481 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:50.991290 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:51.004203 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:51.005829 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:51.016402 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1477/2964] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:28:52.249696 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:52.251268 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:52.262262 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd39c51-7434-4603-a0d0-b77e8cb60518 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:52.266067 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db335e266cd14d45868e6875a9ea34d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:52.266711 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:52.267036 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:52.267310 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:52.269149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:52.331756 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:52.333187 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:52.342858 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1488/2965] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:28:58.508635 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:58.511340 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:58.525988 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8604a8a-229d-4e87-8273-89b2e67476da service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:58.531970 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:58.533339 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:58.533771 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:58.534157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] PATH_INFO: `/v3/users/1311236603b34138ac25eb3d1d98929e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:58.536219 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1311236603b34138ac25eb3d1d98929e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:58.539480 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:58.591884 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1311236603b34138ac25eb3d1d98929e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.592550 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1311236603b34138ac25eb3d1d98929e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.599804 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1311236603b34138ac25eb3d1d98929e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.600478 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1311236603b34138ac25eb3d1d98929e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.612484 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1311236603b34138ac25eb3d1d98929e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.628109 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1478/2966] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:28:58 2022] DELETE /identity/v3/users/1311236603b34138ac25eb3d1d98929e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:28:58.812436 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:58.819721 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:58.823944 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c9492-6c4b-44b8-9367-dfb7518be92c service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:58.884815 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:58.886024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:58.886024 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:58.886314 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] PATH_INFO: `/v3/projects/9c4c13c5b10447a98c6066552f38015b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:58.887575 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4c13c5b10447a98c6066552f38015b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:58.889523 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:58.937626 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4c13c5b10447a98c6066552f38015b', 'request_id': 'req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.944531 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4c13c5b10447a98c6066552f38015b', 'request_id': 'req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.950630 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4c13c5b10447a98c6066552f38015b', 'request_id': 'req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.966244 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] The token cache is being invalidate because project 9c4c13c5b10447a98c6066552f38015b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:28:58.966530 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.967184 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:58.968758 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1489/2967] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:28:58 2022] DELETE /identity/v3/projects/9c4c13c5b10447a98c6066552f38015b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:28:58.973361 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:58.974946 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:58.989437 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300875a-cc1e-4c44-be3c-9d2923668539 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:59.026780 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:59.052985 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:59.053874 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:59.054154 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:59.054424 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] PATH_INFO: `/v3/users/5441bc5121484ac5a706705e8fa34cfe` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:59.055742 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5441bc5121484ac5a706705e8fa34cfe)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:59.057712 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:59.114183 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5441bc5121484ac5a706705e8fa34cfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.114964 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5441bc5121484ac5a706705e8fa34cfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.122412 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5441bc5121484ac5a706705e8fa34cfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.123186 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5441bc5121484ac5a706705e8fa34cfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.136890 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5441bc5121484ac5a706705e8fa34cfe'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.152526 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1479/2968] 10.222.0.32 () {58 vars in 1223 bytes} [Wed May 25 15:28:58 2022] DELETE /identity/v3/users/5441bc5121484ac5a706705e8fa34cfe => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:28:59.341280 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:28:59.342786 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:28:59.357609 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c438a9-a1c1-4ba7-a01a-703b7f1b1eb9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:28:59.416479 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73561edcff5e424d9ae49ab57406b79a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9157b1edcbb3430198b55709b5ddfe19', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:28:59.417349 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:28:59.417628 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:28:59.417895 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] PATH_INFO: `/v3/projects/89e825a0ce8e4bb583196114cf33bb53` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:28:59.419162 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e825a0ce8e4bb583196114cf33bb53)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:28:59.421082 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:28:59.460929 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:28:59.473572 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e825a0ce8e4bb583196114cf33bb53', 'request_id': 'req-95d7c13f-08f4-4433-b556-837ec97546c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.483092 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e825a0ce8e4bb583196114cf33bb53', 'request_id': 'req-95d7c13f-08f4-4433-b556-837ec97546c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.489715 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e825a0ce8e4bb583196114cf33bb53', 'request_id': 'req-95d7c13f-08f4-4433-b556-837ec97546c3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.506860 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] The token cache is being invalidate because project 89e825a0ce8e4bb583196114cf33bb53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 15:28:59.507149 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.508071 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-95d7c13f-08f4-4433-b556-837ec97546c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 15:28:59.510533 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1490/2969] 10.222.0.32 () {58 vars in 1232 bytes} [Wed May 25 15:28:59 2022] DELETE /identity/v3/projects/89e825a0ce8e4bb583196114cf33bb53 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:29:09.149157 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:29:09.502867 dv-cinder-815746-6-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:30:45.599865 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:30:45.601553 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:30:45.674769 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b1487-2420-47b5-b9b0-3b6ca70560f8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:30:45.737165 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c1699703bdb4d2e8dac40a28792cd32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ab7ba398fd04278a753acc66664f6e1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:30:45.737810 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:30:45.738084 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:30:45.738354 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:30:45.740331 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:30:45.751625 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:30:45.753252 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297a8fe-233f-46e3-81ee-fb58f5ce4dc8 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:30:45.763153 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1480/2970] 10.222.0.32 () {60 vars in 1365 bytes} [Wed May 25 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:30:55.759181 dv-cinder-815746-6-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=58 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}